1
|
Wang FA, Li Y, Zeng T. Deep Learning of radiology-genomics integration for computational oncology: A mini review. Comput Struct Biotechnol J 2024; 23:2708-2716. [PMID: 39035833 PMCID: PMC11260400 DOI: 10.1016/j.csbj.2024.06.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2024] [Revised: 06/18/2024] [Accepted: 06/18/2024] [Indexed: 07/23/2024] Open
Abstract
In the field of computational oncology, patient status is often assessed using radiology-genomics, which includes two key technologies and data, such as radiology and genomics. Recent advances in deep learning have facilitated the integration of radiology-genomics data, and even new omics data, significantly improving the robustness and accuracy of clinical predictions. These factors are driving artificial intelligence (AI) closer to practical clinical applications. In particular, deep learning models are crucial in identifying new radiology-genomics biomarkers and therapeutic targets, supported by explainable AI (xAI) methods. This review focuses on recent developments in deep learning for radiology-genomics integration, highlights current challenges, and outlines some research directions for multimodal integration and biomarker discovery of radiology-genomics or radiology-omics that are urgently needed in computational oncology.
Collapse
Affiliation(s)
- Feng-ao Wang
- Key Laboratory of Systems Health Science of Zhejiang Province, School of Life Science, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou 310024, China
| | - Yixue Li
- Key Laboratory of Systems Health Science of Zhejiang Province, School of Life Science, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou 310024, China
- Guangzhou National Laboratory, Guangzhou, China
- GMU-GIBH Joint School of Life Sciences, The Guangdong-Hong Kong-Macau Joint Laboratory for Cell Fate Regulation and Diseases, Guangzhou Laboratory, Guangzhou Medical University, Guangzhou, China
| | - Tao Zeng
- Guangzhou National Laboratory, Guangzhou, China
- GMU-GIBH Joint School of Life Sciences, The Guangdong-Hong Kong-Macau Joint Laboratory for Cell Fate Regulation and Diseases, Guangzhou Laboratory, Guangzhou Medical University, Guangzhou, China
| |
Collapse
|
2
|
Riaz IB, Khan MA, Haddad TC. Potential application of artificial intelligence in cancer therapy. Curr Opin Oncol 2024; 36:437-448. [PMID: 39007164 DOI: 10.1097/cco.0000000000001068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/16/2024]
Abstract
PURPOSE OF REVIEW This review underscores the critical role and challenges associated with the widespread adoption of artificial intelligence in cancer care to enhance disease management, streamline clinical processes, optimize data retrieval of health information, and generate and synthesize evidence. RECENT FINDINGS Advancements in artificial intelligence models and the development of digital biomarkers and diagnostics are applicable across the cancer continuum from early detection to survivorship care. Additionally, generative artificial intelligence has promised to streamline clinical documentation and patient communications, generate structured data for clinical trial matching, automate cancer registries, and facilitate advanced clinical decision support. Widespread adoption of artificial intelligence has been slow because of concerns about data diversity and data shift, model reliability and algorithm bias, legal oversight, and high information technology and infrastructure costs. SUMMARY Artificial intelligence models have significant potential to transform cancer care. Efforts are underway to deploy artificial intelligence models in the cancer practice, evaluate their clinical impact, and enhance their fairness and explainability. Standardized guidelines for the ethical integration of artificial intelligence models in cancer care pathways and clinical operations are needed. Clear governance and oversight will be necessary to gain trust in artificial intelligence-assisted cancer care by clinicians, scientists, and patients.
Collapse
Affiliation(s)
- Irbaz Bin Riaz
- Department of AI and Informatics, Mayo Clinic, Minnesota
- Division of Hematology and Oncology, Mayo Clinic, Phoenix, Arizona
| | | | - Tufia C Haddad
- Department of Oncology, Mayo Clinic, Rochester, Minnesota, USA
| |
Collapse
|
3
|
Zheng L, Yang LX, Liu JY, Jiang Z, Li XW, Pu PP. Correlation and predictive value of pathological complete response and ultrasound characteristic parameters in neoadjuvant chemotherapy for breast. World J Clin Cases 2024; 12:5320-5328. [DOI: 10.12998/wjcc.v12.i23.5320] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/10/2024] [Revised: 05/12/2024] [Accepted: 06/11/2024] [Indexed: 07/05/2024] Open
Abstract
BACKGROUND Breast cancer ranks as one of the most prevalent malignant tumors among women, significantly endangering their health and lives. While radical surgery has been a pivotal method for halting disease progression, it alone is insufficient for enhancing the quality of life for patients.
AIM To investigate the correlation between ultrasound characteristic parameters of breast cancer lesions and clinical efficacy in patients undergoing neoadjuvant chemotherapy (NAC).
METHODS Employing a case-control study design, this research involved 178 breast cancer patients treated with NAC at our hospital from July 2019 to June 2022. According to the Miller-Payne grading system, the pathological response, i.e. efficacy, of the NAC in the initial breast lesion after NAC was evaluated. Of these, 59 patients achieved a pathological complete response (PCR), while 119 did not (non-PCR group). Ultrasound characteristics prior to NAC were compared between these groups, and the association of various factors with NAC efficacy was analyzed using univariate and multivariate approaches.
RESULTS In the PCR group, the incidence of posterior echo attenuation, lesion diameter ≥ 2.0 cm, and Alder blood flow grade ≥ II were significantly lower compared to the non-PCR group (P < 0.05). The area under the curve values for predicting NAC efficacy using posterior echo attenuation, lesion diameter, and Alder grade were 0.604, 0.603, and 0.583, respectively. Also, rates of pathological stage II, lymph node metastasis, vascular invasion, and positive Ki-67 expression were significantly lower in the PCR group (P < 0.05). Logistic regression analysis identified posterior echo attenuation, lesion diameter ≥ 2.0 cm, Alder blood flow grade ≥ II, pathological stage III, vascular invasion, and positive Ki-67 expression as independent predictors of poor response to NAC in breast cancer patients (P < 0.05).
CONCLUSION While ultrasound characteristics such as posterior echo attenuation, lesion diameter ≥ 2.0 cm, and Alder blood flow grade ≥ II exhibit limited predictive value for NAC efficacy, they are significantly associated with poor response to NAC in breast cancer patients.
Collapse
Affiliation(s)
- Lei Zheng
- Department of Breast Surgery, Xingtai People's Hospital, Xingtai 054001, Hebei Province, China
| | - Li-Xian Yang
- Department of Breast Surgery, Xingtai People's Hospital, Xingtai 054001, Hebei Province, China
| | - Jing-Yi Liu
- Department of Breast Surgery, Xingtai People's Hospital, Xingtai 054001, Hebei Province, China
| | - Zhe Jiang
- Department of Medical Imaging, Xingtai People´s Hospital, Xingtai 054001, Hebei Province, China
| | - Xiao-Wei Li
- Department of Breast Surgery, Xingtai People's Hospital, Xingtai 054001, Hebei Province, China
| | - Peng-Peng Pu
- Department of Breast Surgery, Xingtai People's Hospital, Xingtai 054001, Hebei Province, China
| |
Collapse
|
4
|
Shimizu H, Enda K, Koyano H, Shimizu T, Shimodan S, Sato K, Ogawa T, Tanaka S, Iwasaki N, Takahashi D. Bimodal machine learning model for unstable hips in infants: integration of radiographic images with automatically-generated clinical measurements. Sci Rep 2024; 14:17826. [PMID: 39090235 PMCID: PMC11294347 DOI: 10.1038/s41598-024-68484-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/25/2023] [Accepted: 07/24/2024] [Indexed: 08/04/2024] Open
Abstract
Bimodal convolutional neural networks (CNNs) are frequently combined with patient information or several medical images to enhance the diagnostic performance. However, the technologies that integrate automatically generated clinical measurements within the images are scarce. Hence, we developed a bimodal model that produced automatic algorithm for clinical measurement (aaCM) from radiographic images and integrated the model with CNNs. In this multicenter research project, the diagnostic performance of the model was investigated with 813 radiographic hip images of infants at risk of developmental dysplasia of the hips (232 and 581 images of unstable and stable hips, respectively), with the ground truth defined by provocative examinations. The results indicated that the accuracy of aaCM was equal or higher than that of specialists, and the bimodal model showed better diagnostic performance than LightGBM, XGBoost, SVM, and single CNN models. aaCM can provide expert's knowledge in a high level, and our proposed bimodal model has better performance than the state-of-art models.
Collapse
Affiliation(s)
- Hirokazu Shimizu
- Department of Orthopaedic Surgery, Faculty of Medicine and Graduate School of Medicine, Hokkaido University, Sapporo, Hokkaido, Japan
- Department of Cancer Pathology, Faculty of Medicine and Graduate School of Medicine, Hokkaido University, Sapporo, Hokkaido, Japan
| | - Ken Enda
- Department of Cancer Pathology, Faculty of Medicine and Graduate School of Medicine, Hokkaido University, Sapporo, Hokkaido, Japan
| | - Hidenori Koyano
- Graduate School of Biomedical Science and Engineering, Hokkaido University, Sapporo, Hokkaido, Japan
| | - Tomohiro Shimizu
- Department of Orthopaedic Surgery, Faculty of Medicine and Graduate School of Medicine, Hokkaido University, Sapporo, Hokkaido, Japan
| | - Shun Shimodan
- Department of Orthopaedic Surgery, Kushiro City General Hospital, Kushiro, Hokkaido, Japan
| | - Komei Sato
- Department of Orthopaedic Surgery, Faculty of Medicine and Graduate School of Medicine, Hokkaido University, Sapporo, Hokkaido, Japan
| | - Takuya Ogawa
- Department of Orthopaedic Surgery, Faculty of Medicine and Graduate School of Medicine, Hokkaido University, Sapporo, Hokkaido, Japan
| | - Shinya Tanaka
- Department of Cancer Pathology, Faculty of Medicine and Graduate School of Medicine, Hokkaido University, Sapporo, Hokkaido, Japan
- Institute for Chemical Reaction Design and Discovery (WPI-ICReDD), Hokkaido University, Sapporo, Hokkaido, Japan
| | - Norimasa Iwasaki
- Department of Orthopaedic Surgery, Faculty of Medicine and Graduate School of Medicine, Hokkaido University, Sapporo, Hokkaido, Japan
| | - Daisuke Takahashi
- Department of Orthopaedic Surgery, Faculty of Medicine and Graduate School of Medicine, Hokkaido University, Sapporo, Hokkaido, Japan.
| |
Collapse
|
5
|
Waqas A, Tripathi A, Ramachandran RP, Stewart PA, Rasool G. Multimodal data integration for oncology in the era of deep neural networks: a review. Front Artif Intell 2024; 7:1408843. [PMID: 39118787 PMCID: PMC11308435 DOI: 10.3389/frai.2024.1408843] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2024] [Accepted: 07/09/2024] [Indexed: 08/10/2024] Open
Abstract
Cancer research encompasses data across various scales, modalities, and resolutions, from screening and diagnostic imaging to digitized histopathology slides to various types of molecular data and clinical records. The integration of these diverse data types for personalized cancer care and predictive modeling holds the promise of enhancing the accuracy and reliability of cancer screening, diagnosis, and treatment. Traditional analytical methods, which often focus on isolated or unimodal information, fall short of capturing the complex and heterogeneous nature of cancer data. The advent of deep neural networks has spurred the development of sophisticated multimodal data fusion techniques capable of extracting and synthesizing information from disparate sources. Among these, Graph Neural Networks (GNNs) and Transformers have emerged as powerful tools for multimodal learning, demonstrating significant success. This review presents the foundational principles of multimodal learning including oncology data modalities, taxonomy of multimodal learning, and fusion strategies. We delve into the recent advancements in GNNs and Transformers for the fusion of multimodal data in oncology, spotlighting key studies and their pivotal findings. We discuss the unique challenges of multimodal learning, such as data heterogeneity and integration complexities, alongside the opportunities it presents for a more nuanced and comprehensive understanding of cancer. Finally, we present some of the latest comprehensive multimodal pan-cancer data sources. By surveying the landscape of multimodal data integration in oncology, our goal is to underline the transformative potential of multimodal GNNs and Transformers. Through technological advancements and the methodological innovations presented in this review, we aim to chart a course for future research in this promising field. This review may be the first that highlights the current state of multimodal modeling applications in cancer using GNNs and transformers, presents comprehensive multimodal oncology data sources, and sets the stage for multimodal evolution, encouraging further exploration and development in personalized cancer care.
Collapse
Affiliation(s)
- Asim Waqas
- Department of Machine Learning, Moffitt Cancer Center, Tampa, FL, United States
- Department of Cancer Epidemiology, Moffitt Cancer Center, Tampa, FL, United States
| | - Aakash Tripathi
- Department of Machine Learning, Moffitt Cancer Center, Tampa, FL, United States
| | - Ravi P. Ramachandran
- Department of Electrical and Computer Engineering, Rowan University, Glassboro, NJ, United States
| | - Paul A. Stewart
- Department of Biostatistics and Bioinformatics, Moffitt Cancer Center, Tampa, FL, United States
| | - Ghulam Rasool
- Department of Machine Learning, Moffitt Cancer Center, Tampa, FL, United States
| |
Collapse
|
6
|
Pahud de Mortanges A, Luo H, Shu SZ, Kamath A, Suter Y, Shelan M, Pöllinger A, Reyes M. Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging. NPJ Digit Med 2024; 7:195. [PMID: 39039248 PMCID: PMC11263688 DOI: 10.1038/s41746-024-01190-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2023] [Accepted: 07/15/2024] [Indexed: 07/24/2024] Open
Abstract
Explainable artificial intelligence (XAI) has experienced a vast increase in recognition over the last few years. While the technical developments are manifold, less focus has been placed on the clinical applicability and usability of systems. Moreover, not much attention has been given to XAI systems that can handle multimodal and longitudinal data, which we postulate are important features in many clinical workflows. In this study, we review, from a clinical perspective, the current state of XAI for multimodal and longitudinal datasets and highlight the challenges thereof. Additionally, we propose the XAI orchestrator, an instance that aims to help clinicians with the synopsis of multimodal and longitudinal data, the resulting AI predictions, and the corresponding explainability output. We propose several desirable properties of the XAI orchestrator, such as being adaptive, hierarchical, interactive, and uncertainty-aware.
Collapse
Affiliation(s)
| | - Haozhe Luo
- ARTORG Center for Biomedical Engineering Research, University of Bern, Bern, Switzerland
| | - Shelley Zixin Shu
- ARTORG Center for Biomedical Engineering Research, University of Bern, Bern, Switzerland
| | - Amith Kamath
- ARTORG Center for Biomedical Engineering Research, University of Bern, Bern, Switzerland
| | - Yannick Suter
- ARTORG Center for Biomedical Engineering Research, University of Bern, Bern, Switzerland
- Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Mohamed Shelan
- Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Alexander Pöllinger
- Department of Diagnostic, Interventional and Pediatric Radiology, Inselspital, Bern University Hospital, Bern, Switzerland
| | - Mauricio Reyes
- ARTORG Center for Biomedical Engineering Research, University of Bern, Bern, Switzerland
- Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| |
Collapse
|
7
|
Li Y, El Habib Daho M, Conze PH, Zeghlache R, Le Boité H, Tadayoni R, Cochener B, Lamard M, Quellec G. A review of deep learning-based information fusion techniques for multimodal medical image classification. Comput Biol Med 2024; 177:108635. [PMID: 38796881 DOI: 10.1016/j.compbiomed.2024.108635] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Revised: 03/18/2024] [Accepted: 05/18/2024] [Indexed: 05/29/2024]
Abstract
Multimodal medical imaging plays a pivotal role in clinical diagnosis and research, as it combines information from various imaging modalities to provide a more comprehensive understanding of the underlying pathology. Recently, deep learning-based multimodal fusion techniques have emerged as powerful tools for improving medical image classification. This review offers a thorough analysis of the developments in deep learning-based multimodal fusion for medical classification tasks. We explore the complementary relationships among prevalent clinical modalities and outline three main fusion schemes for multimodal classification networks: input fusion, intermediate fusion (encompassing single-level fusion, hierarchical fusion, and attention-based fusion), and output fusion. By evaluating the performance of these fusion techniques, we provide insight into the suitability of different network architectures for various multimodal fusion scenarios and application domains. Furthermore, we delve into challenges related to network architecture selection, handling incomplete multimodal data management, and the potential limitations of multimodal fusion. Finally, we spotlight the promising future of Transformer-based multimodal fusion techniques and give recommendations for future research in this rapidly evolving field.
Collapse
Affiliation(s)
- Yihao Li
- LaTIM UMR 1101, Inserm, Brest, France; University of Western Brittany, Brest, France
| | - Mostafa El Habib Daho
- LaTIM UMR 1101, Inserm, Brest, France; University of Western Brittany, Brest, France.
| | | | - Rachid Zeghlache
- LaTIM UMR 1101, Inserm, Brest, France; University of Western Brittany, Brest, France
| | - Hugo Le Boité
- Sorbonne University, Paris, France; Ophthalmology Department, Lariboisière Hospital, AP-HP, Paris, France
| | - Ramin Tadayoni
- Ophthalmology Department, Lariboisière Hospital, AP-HP, Paris, France; Paris Cité University, Paris, France
| | - Béatrice Cochener
- LaTIM UMR 1101, Inserm, Brest, France; University of Western Brittany, Brest, France; Ophthalmology Department, CHRU Brest, Brest, France
| | - Mathieu Lamard
- LaTIM UMR 1101, Inserm, Brest, France; University of Western Brittany, Brest, France
| | | |
Collapse
|
8
|
Zou Y, Xue M, Hossain MI, Zhu Q. Ultrasound and diffuse optical tomography-transformer model for assessing pathological complete response to neoadjuvant chemotherapy in breast cancer. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:076007. [PMID: 39050779 PMCID: PMC11268382 DOI: 10.1117/1.jbo.29.7.076007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/23/2024] [Revised: 06/28/2024] [Accepted: 07/01/2024] [Indexed: 07/27/2024]
Abstract
Significance We evaluate the efficiency of integrating ultrasound (US) and diffuse optical tomography (DOT) images for predicting pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) in breast cancer patients. The ultrasound-diffuse optical tomography (USDOT)-Transformer model represents a significant step toward accurate prediction of pCR, which is critical for personalized treatment planning. Aim We aim to develop and assess the performance of the USDOT-Transformer model, which combines US and DOT images with tumor receptor biomarkers to predict the pCR of breast cancer patients under NAC. Approach We developed the USDOT-Transformer model using a dual-input transformer to process co-registered US and DOT images along with tumor receptor biomarkers. Our dataset comprised imaging data from 60 patients at multiple time points during their chemotherapy treatment. We used fivefold cross-validation to assess the model's performance, comparing its results against a single modality of US or DOT. Results The USDOT-Transformer model demonstrated excellent predictive performance, with a mean area under the receiving characteristic curve of 0.96 (95%CI: 0.93 to 0.99) across the fivefold cross-validation. The integration of US and DOT images significantly enhanced the model's ability to predict pCR, outperforming models that relied on a single imaging modality (0.87 for US and 0.82 for DOT). This performance indicates the potential of advanced deep learning techniques and multimodal imaging data for improving the accuracy (ACC) of pCR prediction. Conclusion The USDOT-Transformer model offers a promising non-invasive approach for predicting pCR to NAC in breast cancer patients. By leveraging the structural and functional information from US and DOT images, the model offers a faster and more reliable tool for personalized treatment planning. Future work will focus on expanding the dataset and refining the model to further improve its accuracy and generalizability.
Collapse
Affiliation(s)
- Yun Zou
- Washington University in St. Louis, Department of Biomedical Engineering, St. Louis, Missouri, United States
| | - Minghao Xue
- Washington University in St. Louis, Department of Biomedical Engineering, St. Louis, Missouri, United States
| | - Md Iqbal Hossain
- Washington University in St. Louis, Imaging Science, St. Louis, Missouri, United States
| | - Quing Zhu
- Washington University in St. Louis, Department of Biomedical Engineering, St. Louis, Missouri, United States
- Washington University School of Medicine, Department of Radiology, St. Louis, Missouri, United States
| |
Collapse
|
9
|
Lo Gullo R, Brunekreef J, Marcus E, Han LK, Eskreis-Winkler S, Thakur SB, Mann R, Groot Lipman K, Teuwen J, Pinker K. AI Applications to Breast MRI: Today and Tomorrow. J Magn Reson Imaging 2024. [PMID: 38581127 DOI: 10.1002/jmri.29358] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Revised: 03/07/2024] [Accepted: 03/09/2024] [Indexed: 04/08/2024] Open
Abstract
In breast imaging, there is an unrelenting increase in the demand for breast imaging services, partly explained by continuous expanding imaging indications in breast diagnosis and treatment. As the human workforce providing these services is not growing at the same rate, the implementation of artificial intelligence (AI) in breast imaging has gained significant momentum to maximize workflow efficiency and increase productivity while concurrently improving diagnostic accuracy and patient outcomes. Thus far, the implementation of AI in breast imaging is at the most advanced stage with mammography and digital breast tomosynthesis techniques, followed by ultrasound, whereas the implementation of AI in breast magnetic resonance imaging (MRI) is not moving along as rapidly due to the complexity of MRI examinations and fewer available dataset. Nevertheless, there is persisting interest in AI-enhanced breast MRI applications, even as the use of and indications of breast MRI continue to expand. This review presents an overview of the basic concepts of AI imaging analysis and subsequently reviews the use cases for AI-enhanced MRI interpretation, that is, breast MRI triaging and lesion detection, lesion classification, prediction of treatment response, risk assessment, and image quality. Finally, it provides an outlook on the barriers and facilitators for the adoption of AI in breast MRI. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 6.
Collapse
Affiliation(s)
- Roberto Lo Gullo
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, New York, USA
| | - Joren Brunekreef
- AI for Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
| | - Eric Marcus
- AI for Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
| | - Lynn K Han
- Weill Cornell Medical College, New York-Presbyterian Hospital, New York City, New York, USA
| | - Sarah Eskreis-Winkler
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, New York, USA
| | - Sunitha B Thakur
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, New York, USA
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York City, New York, USA
| | - Ritse Mann
- AI for Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Kevin Groot Lipman
- AI for Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Jonas Teuwen
- AI for Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Katja Pinker
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, New York, USA
| |
Collapse
|
10
|
Comes MC, Fanizzi A, Bove S, Didonna V, Diotiaiuti S, Fadda F, La Forgia D, Giotta F, Latorre A, Nardone A, Palmiotti G, Ressa CM, Rinaldi L, Rizzo A, Talienti T, Tamborra P, Zito A, Lorusso V, Massafra R. Explainable 3D CNN based on baseline breast DCE-MRI to give an early prediction of pathological complete response to neoadjuvant chemotherapy. Comput Biol Med 2024; 172:108132. [PMID: 38508058 DOI: 10.1016/j.compbiomed.2024.108132] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Revised: 01/29/2024] [Accepted: 02/12/2024] [Indexed: 03/22/2024]
Abstract
BACKGROUND So far, baseline Dynamic Contrast-Enhanced Magnetic Resonance Imaging (DCE-MRI) has played a key role for the application of sophisticated artificial intelligence-based models using Convolutional Neural Networks (CNNs) to extract quantitative imaging information as earlier indicators of pathological Complete Response (pCR) achievement in breast cancer patients treated with neoadjuvant chemotherapy (NAC). However, these models did not exploit the DCE-MRI exams in their full geometry as 3D volume but analysed only few individual slices independently, thus neglecting the depth information. METHOD This study aimed to develop an explainable 3D CNN, which fulfilled the task of pCR prediction before the beginning of NAC, by leveraging the 3D information of post-contrast baseline breast DCE-MRI exams. Specifically, for each patient, the network took in input a 3D sequence containing the tumor region, which was previously automatically identified along the DCE-MRI exam. A visual explanation of the decision-making process of the network was also provided. RESULTS To the best of our knowledge, our proposal is competitive than other models in the field, which made use of imaging data alone, reaching a median AUC value of 81.8%, 95%CI [75.3%; 88.3%], a median accuracy value of 78.7%, 95%CI [74.8%; 82.5%], a median sensitivity value of 69.8%, 95%CI [59.6%; 79.9%] and a median specificity value of 83.3%, 95%CI [82.6%; 84.0%], respectively. The median and CIs were computed according to a 10-fold cross-validation scheme for 5 rounds. CONCLUSION Finally, this proposal holds high potential to support clinicians on non-invasively early pursuing or changing patient-centric NAC pathways.
Collapse
Affiliation(s)
- Maria Colomba Comes
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Annarita Fanizzi
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy.
| | - Samantha Bove
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy.
| | - Vittorio Didonna
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Sergio Diotiaiuti
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Federico Fadda
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Daniele La Forgia
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Francesco Giotta
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Agnese Latorre
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Annalisa Nardone
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Gennaro Palmiotti
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Cosmo Maurizio Ressa
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Lucia Rinaldi
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Alessandro Rizzo
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Tiziana Talienti
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Pasquale Tamborra
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Alfredo Zito
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Vito Lorusso
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Raffaella Massafra
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| |
Collapse
|
11
|
Lo Gullo R, Marcus E, Huayanay J, Eskreis-Winkler S, Thakur S, Teuwen J, Pinker K. Artificial Intelligence-Enhanced Breast MRI: Applications in Breast Cancer Primary Treatment Response Assessment and Prediction. Invest Radiol 2024; 59:230-242. [PMID: 37493391 PMCID: PMC10818006 DOI: 10.1097/rli.0000000000001010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/27/2023]
Abstract
ABSTRACT Primary systemic therapy (PST) is the treatment of choice in patients with locally advanced breast cancer and is nowadays also often used in patients with early-stage breast cancer. Although imaging remains pivotal to assess response to PST accurately, the use of imaging to predict response to PST has the potential to not only better prognostication but also allow the de-escalation or omission of potentially toxic treatment with undesirable adverse effects, the accelerated implementation of new targeted therapies, and the mitigation of surgical delays in selected patients. In response to the limited ability of radiologists to predict response to PST via qualitative, subjective assessments of tumors on magnetic resonance imaging (MRI), artificial intelligence-enhanced MRI with classical machine learning, and in more recent times, deep learning, have been used with promising results to predict response, both before the start of PST and in the early stages of treatment. This review provides an overview of the current applications of artificial intelligence to MRI in assessing and predicting response to PST, and discusses the challenges and limitations of their clinical implementation.
Collapse
Affiliation(s)
- Roberto Lo Gullo
- Department of Radiology, Breast Imaging Service, Memorial Sloan Kettering Cancer Center, 300 E 66 Street, New York, NY 10065, USA
| | - Eric Marcus
- AI for Oncology, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, the Netherlands
| | - Jorge Huayanay
- Department of Radiology, Breast Imaging Service, Memorial Sloan Kettering Cancer Center, 300 E 66 Street, New York, NY 10065, USA
- Department of Radiology, National Institute of Neoplastic Diseases, Lima, Peru
| | - Sarah Eskreis-Winkler
- Department of Radiology, Breast Imaging Service, Memorial Sloan Kettering Cancer Center, 300 E 66 Street, New York, NY 10065, USA
| | - Sunitha Thakur
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, New York, USA
| | - Jonas Teuwen
- Department of Radiology, Breast Imaging Service, Memorial Sloan Kettering Cancer Center, 300 E 66 Street, New York, NY 10065, USA
- Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands
- AI for Oncology, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, the Netherlands
| | - Katja Pinker
- Department of Radiology, Breast Imaging Service, Memorial Sloan Kettering Cancer Center, 300 E 66 Street, New York, NY 10065, USA
| |
Collapse
|
12
|
Yu Y, Wang Z, Wang Q, Su X, Li Z, Wang R, Guo T, Gao W, Wang H, Zhang B. Radiomic model based on magnetic resonance imaging for predicting pathological complete response after neoadjuvant chemotherapy in breast cancer patients. Front Oncol 2024; 13:1249339. [PMID: 38357424 PMCID: PMC10865896 DOI: 10.3389/fonc.2023.1249339] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Accepted: 11/02/2023] [Indexed: 02/16/2024] Open
Abstract
Purpose To establish a model combining radiomic and clinicopathological factors based on magnetic resonance imaging to predict pathological complete response (pCR) after neoadjuvant chemotherapy in breast cancer patients. Method MRI images and clinicopathologic data of 329 eligible breast cancer patients from the Affiliated Hospital of Qingdao University from August 2018 to August 2022 were included in this study. All patients received neoadjuvant chemotherapy (NAC), and imaging examinations were performed before and after NAC. A total of 329 patients were randomly allocated to a training set and a test set at a ratio of 7:3. We mainly studied the following three types of prediction models: radiomic models, clinical models, and clinical-radiomic models. All models were evaluated using subject operating characteristic curve analysis and area under the curve (AUC), decision curve analysis (DCA) and calibration curves. Results The AUCs of the clinical prediction model, independent imaging model and clinical combined imaging model in the training set were 0.864 0.968 and 0.984, and those in the test set were 0.724, 0.754 and 0.877, respectively. According to DCA and calibration curves, the clinical-radiomic model showed good predictive performance in both the training set and the test set, and we found that we had developed a more concise clinical-radiomic nomogram. Conclusion We have developed a clinical-radiomic model by integrating radiomic features and clinical factors to predict pCR after NAC in breast cancer patients, thereby contributing to the personalized treatment of patients.
Collapse
Affiliation(s)
- Yimiao Yu
- Department of Radiation Oncology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Zhibo Wang
- Department of Gastroenterological Surgery, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Qi Wang
- Department of Radiation Oncology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Xiaohui Su
- Department of Galactophore, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Zhenghao Li
- Department of Radiation Oncology, The Affiliated Hospital of Qingdao University, Qingdao, China
- Department of Galactophore, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Ruifeng Wang
- Department of Radiation Oncology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Tianhui Guo
- Department of Radiation Oncology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Wen Gao
- Department of Radiation Oncology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Haiji Wang
- Department of Radiation Oncology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Biyuan Zhang
- Department of Radiation Oncology, The Affiliated Hospital of Qingdao University, Qingdao, China
| |
Collapse
|
13
|
Oyelade ON, Irunokhai EA, Wang H. A twin convolutional neural network with hybrid binary optimizer for multimodal breast cancer digital image classification. Sci Rep 2024; 14:692. [PMID: 38184742 PMCID: PMC10771515 DOI: 10.1038/s41598-024-51329-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2023] [Accepted: 01/03/2024] [Indexed: 01/08/2024] Open
Abstract
There is a wide application of deep learning technique to unimodal medical image analysis with significant classification accuracy performance observed. However, real-world diagnosis of some chronic diseases such as breast cancer often require multimodal data streams with different modalities of visual and textual content. Mammography, magnetic resonance imaging (MRI) and image-guided breast biopsy represent a few of multimodal visual streams considered by physicians in isolating cases of breast cancer. Unfortunately, most studies applying deep learning techniques to solving classification problems in digital breast images have often narrowed their study to unimodal samples. This is understood considering the challenging nature of multimodal image abnormality classification where the fusion of high dimension heterogeneous features learned needs to be projected into a common representation space. This paper presents a novel deep learning approach combining a dual/twin convolutional neural network (TwinCNN) framework to address the challenge of breast cancer image classification from multi-modalities. First, modality-based feature learning was achieved by extracting both low and high levels features using the networks embedded with TwinCNN. Secondly, to address the notorious problem of high dimensionality associated with the extracted features, binary optimization method is adapted to effectively eliminate non-discriminant features in the search space. Furthermore, a novel method for feature fusion is applied to computationally leverage the ground-truth and predicted labels for each sample to enable multimodality classification. To evaluate the proposed method, digital mammography images and digital histopathology breast biopsy samples from benchmark datasets namely MIAS and BreakHis respectively. Experimental results obtained showed that the classification accuracy and area under the curve (AUC) for the single modalities yielded 0.755 and 0.861871 for histology, and 0.791 and 0.638 for mammography. Furthermore, the study investigated classification accuracy resulting from the fused feature method, and the result obtained showed that 0.977, 0.913, and 0.667 for histology, mammography, and multimodality respectively. The findings from the study confirmed that multimodal image classification based on combination of image features and predicted label improves performance. In addition, the contribution of the study shows that feature dimensionality reduction based on binary optimizer supports the elimination of non-discriminant features capable of bottle-necking the classifier.
Collapse
Affiliation(s)
- Olaide N Oyelade
- School of Electronics, Electrical Engineering and Computer Science, Queen's University Belfast, Belfast, BT9 SBN, UK.
| | | | - Hui Wang
- School of Electronics, Electrical Engineering and Computer Science, Queen's University Belfast, Belfast, BT9 SBN, UK
| |
Collapse
|
14
|
Janse MHA, Janssen LM, van der Velden BHM, Moman MR, Wolters-van der Ben EJM, Kock MCJM, Viergever MA, van Diest PJ, Gilhuijs KGA. Deep Learning-Based Segmentation of Locally Advanced Breast Cancer on MRI in Relation to Residual Cancer Burden: A Multi-Institutional Cohort Study. J Magn Reson Imaging 2023; 58:1739-1749. [PMID: 36928988 DOI: 10.1002/jmri.28679] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2022] [Revised: 02/22/2023] [Accepted: 02/24/2023] [Indexed: 03/18/2023] Open
Abstract
BACKGROUND While several methods have been proposed for automated assessment of breast-cancer response to neoadjuvant chemotherapy on breast MRI, limited information is available about their performance across multiple institutions. PURPOSE To assess the value and robustness of deep learning-derived volumes of locally advanced breast cancer (LABC) on MRI to infer the presence of residual disease after neoadjuvant chemotherapy. STUDY TYPE Retrospective. SUBJECTS Training cohort: 102 consecutive female patients with LABC scheduled for neoadjuvant chemotherapy (NAC) from a single institution (age: 25-73 years). Independent testing cohort: 55 consecutive female patients with LABC from four institutions (age: 25-72 years). FIELD STRENGTH/SEQUENCE Training cohort: single vendor 1.5 T or 3.0 T. Testing cohort: multivendor 3.0 T. Gradient echo dynamic contrast-enhanced sequences. ASSESSMENT A convolutional neural network (nnU-Net) was trained to segment LABC. Based on resulting tumor volumes, an extremely randomized tree model was trained to assess residual cancer burden (RCB)-0/I vs. RCB-II/III. An independent model was developed using functional tumor volume (FTV). Models were tested on an independent testing cohort and response assessment performance and robustness across multiple institutions were assessed. STATISTICAL TESTS The receiver operating characteristic (ROC) was used to calculate the area under the ROC curve (AUC). DeLong's method was used to compare AUCs. Correlations were calculated using Pearson's method. P values <0.05 were considered significant. RESULTS Automated segmentation resulted in a median (interquartile range [IQR]) Dice score of 0.87 (0.62-0.93), with similar volumetric measurements (R = 0.95, P < 0.05). Automated volumetric measurements were significantly correlated with FTV (R = 0.80). Tumor volume-derived from deep learning of DCE-MRI was associated with RCB, yielding an AUC of 0.76 to discriminate between RCB-0/I and RCB-II/III, performing similar to the FTV-based model (AUC = 0.77, P = 0.66). Performance was comparable across institutions (IQR AUC: 0.71-0.84). DATA CONCLUSION Deep learning-based segmentation estimates changes in tumor load on DCE-MRI that are associated with RCB after NAC and is robust against variations between institutions. EVIDENCE LEVEL 2. TECHNICAL EFFICACY Stage 4.
Collapse
Affiliation(s)
- Markus H A Janse
- Image Sciences Institute, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Liselore M Janssen
- Image Sciences Institute, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Bas H M van der Velden
- Image Sciences Institute, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Maaike R Moman
- Image Sciences Institute, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
- Alexander Monro Hospital, Bilthoven, The Netherlands
| | | | - Marc C J M Kock
- Department of Radiology, Albert Schweitzer Hospital, Dordrecht, The Netherlands
| | - Max A Viergever
- Image Sciences Institute, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Paul J van Diest
- Department of Pathology, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Kenneth G A Gilhuijs
- Image Sciences Institute, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
15
|
Elsayed B, Alksas A, Shehata M, Mahmoud A, Zaky M, Alghandour R, Abdelwahab K, Abdelkhalek M, Ghazal M, Contractor S, El-Din Moustafa H, El-Baz A. Exploring Neoadjuvant Chemotherapy, Predictive Models, Radiomic, and Pathological Markers in Breast Cancer: A Comprehensive Review. Cancers (Basel) 2023; 15:5288. [PMID: 37958461 PMCID: PMC10648987 DOI: 10.3390/cancers15215288] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2023] [Revised: 10/30/2023] [Accepted: 11/01/2023] [Indexed: 11/15/2023] Open
Abstract
Breast cancer retains its position as the most prevalent form of malignancy among females on a global scale. The careful selection of appropriate treatment for each patient holds paramount importance in effectively managing breast cancer. Neoadjuvant chemotherapy (NACT) plays a pivotal role in the comprehensive treatment of this disease. Administering chemotherapy before surgery, NACT becomes a powerful tool in reducing tumor size, potentially enabling fewer invasive surgical procedures and even rendering initially inoperable tumors amenable to surgery. However, a significant challenge lies in the varying responses exhibited by different patients towards NACT. To address this challenge, researchers have focused on developing prediction models that can identify those who would benefit from NACT and those who would not. Such models have the potential to reduce treatment costs and contribute to a more efficient and accurate management of breast cancer. Therefore, this review has two objectives: first, to identify the most effective radiomic markers correlated with NACT response, and second, to explore whether integrating radiomic markers extracted from radiological images with pathological markers can enhance the predictive accuracy of NACT response. This review will delve into addressing these research questions and also shed light on the emerging research direction of leveraging artificial intelligence techniques for predicting NACT response, thereby shaping the future landscape of breast cancer treatment.
Collapse
Affiliation(s)
- Basma Elsayed
- Biomedical Engineering Program, Faculty of Engineering, Mansoura University, Mansoura 35516, Egypt;
| | - Ahmed Alksas
- Department of Bioengineering, University of Louisville, Louisville, KY 40292, USA; (A.A.); (M.S.); (A.M.)
| | - Mohamed Shehata
- Department of Bioengineering, University of Louisville, Louisville, KY 40292, USA; (A.A.); (M.S.); (A.M.)
| | - Ali Mahmoud
- Department of Bioengineering, University of Louisville, Louisville, KY 40292, USA; (A.A.); (M.S.); (A.M.)
| | - Mona Zaky
- Diagnostic Radiology Department, Faculty of Medicine, Mansoura University, Mansoura 35516, Egypt;
| | - Reham Alghandour
- Medical Oncology Department, Mansoura Oncology Center, Mansoura University, Mansoura 35516, Egypt;
| | - Khaled Abdelwahab
- Surgical Oncology Department, Mansoura Oncology Center, Mansoura University, Mansoura 35516, Egypt; (K.A.); (M.A.)
| | - Mohamed Abdelkhalek
- Surgical Oncology Department, Mansoura Oncology Center, Mansoura University, Mansoura 35516, Egypt; (K.A.); (M.A.)
| | - Mohammed Ghazal
- Electrical, Computer, and Biomedical Engineering Department, Abu Dhabi University, Abu Dhabi 59911, United Arab Emirates;
| | - Sohail Contractor
- Department of Radiology, University of Louisville, Louisville, KY 40202, USA;
| | | | - Ayman El-Baz
- Department of Bioengineering, University of Louisville, Louisville, KY 40292, USA; (A.A.); (M.S.); (A.M.)
| |
Collapse
|
16
|
Łazarczyk A, Streb J, Glajcar A, Streb-Smoleń A, Hałubiec P, Wcisło K, Laskowicz Ł, Hodorowicz-Zaniewska D, Szpor J. Dendritic Cell Subpopulations Are Associated with Prognostic Characteristics of Breast Cancer after Neoadjuvant Chemotherapy-An Observational Study. Int J Mol Sci 2023; 24:15817. [PMID: 37958800 PMCID: PMC10648319 DOI: 10.3390/ijms242115817] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2023] [Revised: 10/29/2023] [Accepted: 10/30/2023] [Indexed: 11/15/2023] Open
Abstract
Breast cancer (BC) is the most prevalent malignancy in women and researchers have strived to develop optimal strategies for its diagnosis and management. Neoadjuvant chemotherapy (NAC), which reduces tumor size, risk of metastasis and patient mortality, often also allows for a de-escalation of breast and axillary surgery. Nonetheless, complete pathological response (pCR) is achieved in no more than 40% of patients who underwent NAC. Dendritic cells (DCs) are professional antigen-presenting cells present in the tumor microenvironment. The multitude of their subtypes was shown to be associated with the pathological and clinical characteristics of BC, but it was not evaluated in BC tissue after NAC. We found that highe r densities of CD123+ plasmacytoid DCs (pDCs) were present in tumors that did not show pCR and had a higher residual cancer burden (RCB) score and class. They were of higher stage and grade and more frequently HER2-negative. The density of CD123+ pCDs was an independent predictor of pCR in the studied group. DC-LAMP+ mature DCs (mDCs) were also related to characteristics of clinical relevance (i.e., pCR, RCB, and nuclear grade), although no clear trends were identified. We conclude that CD123+ pDCs are candidates for a novel biomarker of BC response to NAC.
Collapse
Affiliation(s)
- Agnieszka Łazarczyk
- Department of Pathomorphology, Jagiellonian University Medical College, 31-501 Cracow, Poland (J.S.)
| | - Joanna Streb
- Department of Oncology, Jagiellonian University Medical College, 31-501 Cracow, Poland
- University Centre of Breast Disease, University Hospital, 31-501 Cracow, Poland
| | - Anna Glajcar
- Department of Pathomorphology, University Hospital, 30-688 Cracow, Poland
| | - Anna Streb-Smoleń
- Department of Oncology, Maria Sklodowska-Curie National Research Institute of Oncology, 31-115 Cracow, Poland
| | - Przemysław Hałubiec
- Doctoral School of Medical and Health Sciences, Jagiellonian University Medical College, 31-530 Cracow, Poland
| | - Kacper Wcisło
- Department of Pathomorphology, Jagiellonian University Medical College, 31-501 Cracow, Poland (J.S.)
- Department of Pathomorphology, University Hospital, 30-688 Cracow, Poland
| | - Łukasz Laskowicz
- Clinical Department of Gynecology and Gynecological Oncology, University Hospital, 30-688 Cracow, Poland
| | - Diana Hodorowicz-Zaniewska
- General, Oncological and Gastrointestinal Surgery, Jagiellonian University Medical College, 31-501 Cracow, Poland;
- Department of General Surgery, University Hospital, 31-501 Cracow, Poland
| | - Joanna Szpor
- Department of Pathomorphology, Jagiellonian University Medical College, 31-501 Cracow, Poland (J.S.)
- University Centre of Breast Disease, University Hospital, 31-501 Cracow, Poland
- Department of Pathomorphology, University Hospital, 30-688 Cracow, Poland
| |
Collapse
|
17
|
Kita K, Fujimori T, Suzuki Y, Kanie Y, Takenaka S, Kaito T, Taki T, Ukon Y, Furuya M, Saiwai H, Nakajima N, Sugiura T, Ishiguro H, Kamatani T, Tsukazaki H, Sakai Y, Takami H, Tateiwa D, Hashimoto K, Wataya T, Nishigaki D, Sato J, Hoshiyama M, Tomiyama N, Okada S, Kido S. Bimodal artificial intelligence using TabNet for differentiating spinal cord tumors-Integration of patient background information and images. iScience 2023; 26:107900. [PMID: 37766987 PMCID: PMC10520519 DOI: 10.1016/j.isci.2023.107900] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2022] [Revised: 02/18/2023] [Accepted: 09/08/2023] [Indexed: 09/29/2023] Open
Abstract
We proposed a bimodal artificial intelligence that integrates patient information with images to diagnose spinal cord tumors. Our model combines TabNet, a state-of-the-art deep learning model for tabular data for patient information, and a convolutional neural network for images. As training data, we collected 259 spinal tumor patients (158 for schwannoma and 101 for meningioma). We compared the performance of the image-only unimodal model, table-only unimodal model, bimodal model using a gradient-boosting decision tree, and bimodal model using TabNet. Our proposed bimodal model using TabNet performed best (area under the receiver-operating characteristic curve [AUROC]: 0.91) in the training data and significantly outperformed the physicians' performance. In the external validation using 62 cases from the other two facilities, our bimodal model showed an AUROC of 0.92, proving the robustness of the model. The bimodal analysis using TabNet was effective for differentiating spinal tumors.
Collapse
Affiliation(s)
- Kosuke Kita
- Osaka University School of Medicine Graduate School of Medicine Diagnostic and Interventional Radiology, Suita, Osaka, Japan
| | - Takahito Fujimori
- Osaka University Graduate School of Medicine Department of Orthopaedic Surgery, Suita, Osaka, Japan
| | - Yuki Suzuki
- Osaka University School of Medicine Graduate School of Medicine Diagnostic and Interventional Radiology, Suita, Osaka, Japan
| | - Yuya Kanie
- Osaka University Graduate School of Medicine Department of Orthopaedic Surgery, Suita, Osaka, Japan
| | - Shota Takenaka
- Osaka University Graduate School of Medicine Department of Orthopaedic Surgery, Suita, Osaka, Japan
| | - Takashi Kaito
- Osaka University Graduate School of Medicine Department of Orthopaedic Surgery, Suita, Osaka, Japan
| | - Takuyu Taki
- Department of Neurosurgery, Iseikai Hospital, Osaka, Osaka, Japan
| | - Yuichiro Ukon
- Osaka University Graduate School of Medicine Department of Orthopaedic Surgery, Suita, Osaka, Japan
| | | | - Hirokazu Saiwai
- Department of Orthopedic Surgery, Graduate School of Medical Sciences, Kyusyu University, Higashi, Fukuoka, Japan
| | - Nozomu Nakajima
- Japanese Red Cross Society Himeji Hospital, Himeji, Hyogo, Japan
| | - Tsuyoshi Sugiura
- General Incorporated Foundation Sumitomo Hospital, Osaka, Osaka, Japan
| | - Hiroyuki Ishiguro
- National Hospital Organization Osaka National Hospital, Osaka, Osaka, Japan
| | | | | | | | - Haruna Takami
- Osaka International Cancer Institute, Osaka, Osaka, Japan
| | | | | | - Tomohiro Wataya
- Osaka University School of Medicine Graduate School of Medicine Diagnostic and Interventional Radiology, Suita, Osaka, Japan
| | - Daiki Nishigaki
- Osaka University School of Medicine Graduate School of Medicine Diagnostic and Interventional Radiology, Suita, Osaka, Japan
| | - Junya Sato
- Osaka University School of Medicine Graduate School of Medicine Diagnostic and Interventional Radiology, Suita, Osaka, Japan
| | | | - Noriyuki Tomiyama
- Osaka University School of Medicine Graduate School of Medicine Diagnostic and Interventional Radiology, Suita, Osaka, Japan
| | - Seiji Okada
- Osaka University Graduate School of Medicine Department of Orthopaedic Surgery, Suita, Osaka, Japan
| | - Shoji Kido
- Osaka University School of Medicine Graduate School of Medicine Diagnostic and Interventional Radiology, Suita, Osaka, Japan
| |
Collapse
|
18
|
Luo J, Li X, Wei KL, Chen G, Xiong DD. Advances in the application of computational pathology in diagnosis, immunomicroenvironment recognition, and immunotherapy evaluation of breast cancer: a narrative review. J Cancer Res Clin Oncol 2023; 149:12535-12542. [PMID: 37389595 DOI: 10.1007/s00432-023-05002-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Accepted: 06/15/2023] [Indexed: 07/01/2023]
Abstract
BACKGROUND Breast cancer (BC) is a prevalent and highly lethal malignancy affecting women worldwide. Immunotherapy has emerged as a promising therapeutic strategy for BC, offering potential improvements in patient survival. Neoadjuvant therapy (NAT) has also gained significant clinical traction. With the advancement of computer technology, Artificial Intelligence (AI) has been increasingly applied in pathology research, expanding and redefining the scope of the field. This narrative review aims to provide a comprehensive overview of the current literature on the application of computational pathology in BC, specifically focusing on diagnosis, immune microenvironment recognition, and the evaluation of immunotherapy and NAT response. METHODS A thorough examination of relevant literature was conducted, focusing on studies investigating the role of computational pathology in BC diagnosis, immune microenvironment recognition, and immunotherapy and NAT assessment. RESULTS The application of computational pathology has shown significant potential in BC management. AI-based techniques enable improved diagnosis and classification of BC subtypes, enhance the identification and characterization of the immune microenvironment, and facilitate the evaluation of immunotherapy and NAT response. However, challenges related to data quality, standardization, and algorithm development still need to be addressed. CONCLUSION The integration of computational pathology and AI has transformative implications for BC patient care. By leveraging AI-based technologies, clinicians can make more informed decisions in diagnosis, treatment planning, and therapeutic response assessment. Future research should focus on refining AI algorithms, addressing technical challenges, and conducting large-scale clinical validation studies to facilitate the translation of computational pathology into routine clinical practice for BC patients.
Collapse
Affiliation(s)
- Jie Luo
- Department of Oncology, The Second Affiliated Hospital of Guangxi Medical University, Nanning, 530007, Guangxi, People's Republic of China
| | - Xia Li
- Department of Pathology, The First Affiliated Hospital of Guangxi Medical University, Nanning, 530021, Guangxi, People's Republic of China
| | - Kang-Lai Wei
- Department of Pathology, The Second Affiliated Hospital of Guangxi Medical University, Nanning, 530007, Guangxi, People's Republic of China
| | - Gang Chen
- Department of Pathology, The First Affiliated Hospital of Guangxi Medical University, Nanning, 530021, Guangxi, People's Republic of China
| | - Dan-Dan Xiong
- Department of Pathology, The First Affiliated Hospital of Guangxi Medical University, Nanning, 530021, Guangxi, People's Republic of China.
| |
Collapse
|
19
|
Park S, Kim JH, Cha YK, Chung MJ, Woo JH, Park S. Application of Machine Learning Algorithm in Predicting Axillary Lymph Node Metastasis from Breast Cancer on Preoperative Chest CT. Diagnostics (Basel) 2023; 13:2953. [PMID: 37761320 PMCID: PMC10528867 DOI: 10.3390/diagnostics13182953] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Revised: 09/05/2023] [Accepted: 09/13/2023] [Indexed: 09/29/2023] Open
Abstract
Axillary lymph node (ALN) status is one of the most critical prognostic factors in patients with breast cancer. However, ALN evaluation with contrast-enhanced CT (CECT) has been challenging. Machine learning (ML) is known to show excellent performance in image recognition tasks. The purpose of our study was to evaluate the performance of the ML algorithm for predicting ALN metastasis by combining preoperative CECT features of both ALN and primary tumor. This was a retrospective single-institutional study of a total of 266 patients with breast cancer who underwent preoperative chest CECT. Random forest (RF), extreme gradient boosting (XGBoost), and neural network (NN) algorithms were used. Statistical analysis and recursive feature elimination (RFE) were adopted as feature selection for ML. The best ML-based ALN prediction model for breast cancer was NN with RFE, which achieved an AUROC of 0.76 ± 0.11 and an accuracy of 0.74 ± 0.12. By comparing NN with RFE model performance with and without ALN features from CECT, NN with RFE model with ALN features showed better performance at all performance evaluations, which indicated the effect of ALN features. Through our study, we were able to demonstrate that the ML algorithm could effectively predict the final diagnosis of ALN metastases from CECT images of the primary tumor and ALN. This suggests that ML has the potential to differentiate between benign and malignant ALNs.
Collapse
Affiliation(s)
- Soyoung Park
- Department of Health Sciences and Technology, SAIHST, Sungkyunkwan University, Seoul 06351, Republic of Korea; (S.P.); (S.P.)
| | - Jong Hee Kim
- Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul 06351, Republic of Korea; (J.H.K.); (J.H.W.)
| | - Yoon Ki Cha
- Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul 06351, Republic of Korea; (J.H.K.); (J.H.W.)
| | - Myung Jin Chung
- Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul 06351, Republic of Korea; (J.H.K.); (J.H.W.)
| | - Jung Han Woo
- Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul 06351, Republic of Korea; (J.H.K.); (J.H.W.)
| | - Subin Park
- Department of Health Sciences and Technology, SAIHST, Sungkyunkwan University, Seoul 06351, Republic of Korea; (S.P.); (S.P.)
| |
Collapse
|
20
|
Verma M, Abdelrahman L, Collado-Mesa F, Abdel-Mottaleb M. Multimodal Spatiotemporal Deep Learning Framework to Predict Response of Breast Cancer to Neoadjuvant Systemic Therapy. Diagnostics (Basel) 2023; 13:2251. [PMID: 37443648 DOI: 10.3390/diagnostics13132251] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Revised: 06/20/2023] [Accepted: 06/27/2023] [Indexed: 07/15/2023] Open
Abstract
Current approaches to breast cancer therapy include neoadjuvant systemic therapy (NST). The efficacy of NST is measured by pathologic complete response (pCR). A patient who attains pCR has significantly enhanced disease-free survival progress. The accurate prediction of pCR in response to a given treatment regimen could increase the likelihood of achieving pCR and prevent toxicities caused by treatments that are not effective. Th early prediction of response to NST can increase the likelihood of survival and help with decisions regarding breast-conserving surgery. An automated NST prediction framework that is able to precisely predict which patient undergoing NST will achieve a pathological complete response (pCR) at an early stage of treatment is needed. Here, we propose an end-to-end efficient multimodal spatiotemporal deep learning framework (deep-NST) framework to predict the outcome of NST prior or at an early stage of treatment. The deep-NST model incorporates imaging data captured at different timestamps of NST regimens, a tumor's molecular data, and a patient's demographic data. The efficacy of the proposed work is validated on the publicly available ISPY-1 dataset, in terms of accuracy, area under the curve (AUC), and computational complexity. In addition, seven ablation experiments were carried out to evaluate the impact of each design module in the proposed work. The experimental results show that the proposed framework performs significantly better than other recent methods.
Collapse
Affiliation(s)
- Monu Verma
- Department of Electrical and Computer Engineering, University of Miami, Miami, FL 33146, USA
| | | | - Fernando Collado-Mesa
- Department of Radiology, Miller School of Medicine, University of Miami, Miami, FL 33146, USA
| | - Mohamed Abdel-Mottaleb
- Department of Electrical and Computer Engineering, University of Miami, Miami, FL 33146, USA
| |
Collapse
|
21
|
Zhou Z, Adrada BE, Candelaria RP, Elshafeey NA, Boge M, Mohamed RM, Pashapoor S, Sun J, Xu Z, Panthi B, Son JB, Guirguis MS, Patel MM, Whitman GJ, Moseley TW, Scoggins ME, White JB, Litton JK, Valero V, Hunt KK, Tripathy D, Yang W, Wei P, Yam C, Pagel MD, Rauch GM, Ma J. Predicting pathological complete response to neoadjuvant systemic therapy for triple-negative breast cancers using deep learning on multiparametric MRIs. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38083160 DOI: 10.1109/embc40787.2023.10340987] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
We trained and validated a deep learning model that can predict the treatment response to neoadjuvant systemic therapy (NAST) for patients with triple negative breast cancer (TNBC). Dynamic contrast enhanced (DCE) MRI and diffusion-weighted imaging (DWI) of the pre-treatment (baseline) and after four cycles (C4) of doxorubicin/cyclophosphamide treatment were used as inputs to the model for prediction of pathologic complete response (pCR). Based on the standard pCR definition that includes disease status in either breast or axilla, the model achieved areas under the receiver operating characteristic curves (AUCs) of 0.96 ± 0.05, 0.78 ± 0.09, 0.88 ± 0.02, and 0.76 ± 0.03, for the training, validation, testing, and prospective testing groups, respectively. For the pCR status of breast only, the retrained model achieved prediction AUCs of 0.97 ± 0.04, 0.82 ± 0.10, 0.86 ± 0.03, and 0.83 ± 0.02, for the training, validation, testing, and prospective testing groups, respectively. Thus, the developed deep learning model is highly promising for predicting the treatment response to NAST of TNBC.Clinical Relevance- Deep learning based on serial and multiparametric MRIs can potentially distinguish TNBC patients with pCR from non-pCR at the early stage of neoadjuvant systemic therapy, potentially enabling more personalized treatment of TNBC patients.
Collapse
|
22
|
Hao X, Xu H, Zhao N, Yu T, Hamalainen T, Cong F. Predicting pathological complete response based on weakly and semi-supervised joint learning from breast cancer MRI. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38083773 DOI: 10.1109/embc40787.2023.10340081] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Neoadjuvant chemotherapy (NAC) is the standard treatment for breast cancer patients. Patients achieving complete pathological response (pCR) after NAC usually have a good prognosis. However, automatic pCR prediction has been a challenging problem due to lacking well annotations in 3D MRI. Thus far, unifying different annotation information to predict the tumor's early response to NAC has not been systematically addressed. This paper proposes a weakly and semi-supervised joint learning method that integrates attentional features from multi-parametric MRI with radiomic features for predicting pCR to NAC in breast cancer patients. The attention-based multi-instance learning (MIL) is first developed to generate informative MRI bag-level features and mine key instances. The mean-teacher framework is then employed to segment tumor regions in a semi-supervised setting for extracting radiomic features. We perform experiments on 442 patients' data and show that our method achieves an AUC value of 0.85 in pCR prediction, which is superior to comparative methods. It is also shown that learning from multi-parametric MRI outperforms that of single-parameter MRI in pCR prediction.
Collapse
|
23
|
Peterson JR, Cole JA, Pfeiffer JR, Norris GH, Zhang Y, Lopez-Ramos D, Pandey T, Biancalana M, Esslinger HR, Antony AK, Takiar V. Novel computational biology modeling system can accurately forecast response to neoadjuvant therapy in early breast cancer. Breast Cancer Res 2023; 25:54. [PMID: 37165441 PMCID: PMC10170712 DOI: 10.1186/s13058-023-01654-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2022] [Accepted: 05/02/2023] [Indexed: 05/12/2023] Open
Abstract
BACKGROUND Generalizable population-based studies are unable to account for individual tumor heterogeneity that contributes to variability in a patient's response to physician-chosen therapy. Although molecular characterization of tumors has advanced precision medicine, in early-stage and locally advanced breast cancer patients, predicting a patient's response to neoadjuvant therapy (NAT) remains a gap in current clinical practice. Here, we perform a study in an independent cohort of early-stage and locally advanced breast cancer patients to forecast tumor response to NAT and assess the stability of a previously validated biophysical simulation platform. METHODS A single-blinded study was performed using a retrospective database from a single institution (9/2014-12/2020). Patients included: ≥ 18 years with breast cancer who completed NAT, with pre-treatment dynamic contrast enhanced magnetic resonance imaging. Demographics, chemotherapy, baseline (pre-treatment) MRI and pathologic data were input into the TumorScope Predict (TS) biophysical simulation platform to generate predictions. Primary outcomes included predictions of pathological complete response (pCR) versus residual disease (RD) and final volume for each tumor. For validation, post-NAT predicted pCR and tumor volumes were compared to actual pathological assessment and MRI-assessed volumes. Predicted pCR was pre-defined as residual tumor volume ≤ 0.01 cm3 (≥ 99.9% reduction). RESULTS The cohort consisted of eighty patients; 36 Caucasian and 40 African American. Most tumors were high-grade (54.4% grade 3) invasive ductal carcinomas (90.0%). Receptor subtypes included hormone receptor positive (HR+)/human epidermal growth factor receptor 2 positive (HER2+, 30%), HR+/HER2- (35%), HR-/HER2+ (12.5%) and triple negative breast cancer (TNBC, 22.5%). Simulated tumor volume was significantly correlated with post-treatment radiographic MRI calculated volumes (r = 0.53, p = 1.3 × 10-7, mean absolute error of 6.57%). TS prediction of pCR compared favorably to pathological assessment (pCR: TS n = 28; Path n = 27; RD: TS n = 52; Path n = 53), for an overall accuracy of 91.2% (95% CI: 82.8% - 96.4%; Clopper-Pearson interval). Five-year risk of recurrence demonstrated similar prognostic performance between TS predictions (Hazard ratio (HR): - 1.99; 95% CI [- 3.96, - 0.02]; p = 0.043) and clinically assessed pCR (HR: - 1.76; 95% CI [- 3.75, 0.23]; p = 0.054). CONCLUSION We demonstrated TS ability to simulate and model tumor in vivo conditions in silico and forecast volume response to NAT across breast tumor subtypes.
Collapse
Affiliation(s)
- Joseph R Peterson
- SimBioSys, Inc., 180 N La Salle St. Suite 3250, Chicago, IL, 60601, USA.
| | - John A Cole
- SimBioSys, Inc., 180 N La Salle St. Suite 3250, Chicago, IL, 60601, USA
| | - John R Pfeiffer
- SimBioSys, Inc., 180 N La Salle St. Suite 3250, Chicago, IL, 60601, USA
| | - Gregory H Norris
- SimBioSys, Inc., 180 N La Salle St. Suite 3250, Chicago, IL, 60601, USA
| | - Yuhan Zhang
- SimBioSys, Inc., 180 N La Salle St. Suite 3250, Chicago, IL, 60601, USA
| | - Dorys Lopez-Ramos
- SimBioSys, Inc., 180 N La Salle St. Suite 3250, Chicago, IL, 60601, USA
| | - Tushar Pandey
- SimBioSys, Inc., 180 N La Salle St. Suite 3250, Chicago, IL, 60601, USA
| | | | - Hope R Esslinger
- Department of Radiation Oncology, University of Cincinnati, College of Medicine, Cincinnati, OH, USA
| | - Anuja K Antony
- SimBioSys, Inc., 180 N La Salle St. Suite 3250, Chicago, IL, 60601, USA
| | - Vinita Takiar
- Department of Radiation Oncology, University of Cincinnati, College of Medicine, Cincinnati, OH, USA
| |
Collapse
|
24
|
Zhang H, Zhuang Y, Xia S, Jiang H. Deep Learning Network with Spatial Attention Module for Detecting Acute Bilirubin Encephalopathy in Newborns Based on Multimodal MRI. Diagnostics (Basel) 2023; 13:diagnostics13091577. [PMID: 37174968 PMCID: PMC10178403 DOI: 10.3390/diagnostics13091577] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Revised: 04/25/2023] [Accepted: 04/26/2023] [Indexed: 05/15/2023] Open
Abstract
BACKGROUND Acute bilirubin encephalopathy (ABE) is a significant cause of neonatal mortality and disability. Early detection and treatment of ABE can prevent the further development of ABE and its long-term complications. Due to the limited classification ability of single-modal magnetic resonance imaging (MRI), this study aimed to validate the classification performance of a new deep learning model based on multimodal MRI images. Additionally, the study evaluated the effect of a spatial attention module (SAM) on improving the model's diagnostic performance in distinguishing ABE. METHODS This study enrolled a total of 97 neonates diagnosed with ABE and 80 neonates diagnosed with hyperbilirubinemia (HB, non-ABE). Each patient underwent three types of multimodal imaging, which included T1-weighted imaging (T1WI), T2-weighted imaging (T2WI), and an apparent diffusion coefficient (ADC) map. A multimodal MRI classification model based on the ResNet18 network with spatial attention modules was built to distinguish ABE from non-ABE. All combinations of the three types of images were used as inputs to test the model's classification performance, and we also analyzed the prediction performance of models with SAMs through comparative experiments. RESULTS The results indicated that the diagnostic performance of the multimodal image combination was better than any single-modal image, and the combination of T1WI and T2WI achieved the best classification performance (accuracy = 0.808 ± 0.069, area under the curve = 0.808 ± 0.057). The ADC images performed the worst among the three modalities' images. Adding spatial attention modules significantly improved the model's classification performance. CONCLUSION Our experiment showed that a multimodal image classification network with spatial attention modules significantly improved the accuracy of ABE classification.
Collapse
Affiliation(s)
- Huan Zhang
- Key Laboratory of Biomedical Engineering of Ministry of Education, Zhejiang University, Hangzhou 310027, China
| | - Yi Zhuang
- Department of Radiology, Affiliated Children's Hospital of Jiangnan University, Wuxi 214036, China
| | - Shunren Xia
- Key Laboratory of Biomedical Engineering of Ministry of Education, Zhejiang University, Hangzhou 310027, China
| | - Haoxiang Jiang
- Department of Radiology, Affiliated Children's Hospital of Jiangnan University, Wuxi 214036, China
| |
Collapse
|
25
|
Huang Y, Zhu T, Zhang X, Li W, Zheng X, Cheng M, Ji F, Zhang L, Yang C, Wu Z, Ye G, Lin Y, Wang K. Longitudinal MRI-based fusion novel model predicts pathological complete response in breast cancer treated with neoadjuvant chemotherapy: a multicenter, retrospective study. EClinicalMedicine 2023; 58:101899. [PMID: 37007742 PMCID: PMC10050775 DOI: 10.1016/j.eclinm.2023.101899] [Citation(s) in RCA: 26] [Impact Index Per Article: 26.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/10/2022] [Revised: 02/17/2023] [Accepted: 02/21/2023] [Indexed: 04/04/2023] Open
Abstract
Background Accurate identification of pCR to neoadjuvant chemotherapy (NAC) is essential for determining appropriate surgery strategy and guiding resection extent in breast cancer. However, a non-invasive tool to predict pCR accurately is lacking. Our study aims to develop ensemble learning models using longitudinal multiparametric MRI to predict pCR in breast cancer. Methods From July 2015 to December 2021, we collected pre-NAC and post-NAC multiparametric MRI sequences per patient. We then extracted 14,676 radiomics and 4096 deep learning features and calculated additional delta-value features. In the primary cohort (n = 409), the inter-class correlation coefficient test, U-test, Boruta and the least absolute shrinkage and selection operator regression were used to select the most significant features for each subtype of breast cancer. Five machine learning classifiers were then developed to predict pCR accurately for each subtype. The ensemble learning strategy was used to integrate the single-modality models. The diagnostic performances of models were evaluated in the three external cohorts (n = 343, 170 and 340, respectively). Findings A total of 1262 patients with breast cancer from four centers were enrolled in this study, and pCR rates were 10.6% (52/491), 54.3% (323/595) and 37.5% (66/176) in HR+/HER2-, HER2+ and TNBC subtype, respectively. Finally, 20, 15 and 13 features were selected to construct the machine learning models in HR+/HER2-, HER2+ and TNBC subtypes, respectively. The multi-Layer Perception (MLP) yields the best diagnostic performances in all subtypes. For the three subtypes, the stacking model integrating pre-, post- and delta-models yielded the highest AUCs of 0.959, 0.974 and 0.958 in the primary cohort, and AUCs of 0.882-0.908, 0.896-0.929 and 0.837-0.901 in the external validation cohorts, respectively. The stacking model had accuracies of 85.0%-88.9%, sensitivities of 80.0%-86.3%, and specificities of 87.4%-91.5% in the external validation cohorts. Interpretation Our study established a novel tool to predict the responses of breast cancer to NAC and achieve excellent performance. The models could help to determine post-NAC surgery strategy for breast cancer. Funding This study is supported by grants from the National Natural Science Foundation of China (82171898, 82103093), the Deng Feng project of high-level hospital construction (DFJHBF202109), the Guangdong Basic and Applied Basic Research Foundation (grant number, 2020A1515010346, 2022A1515012277), the Science and Technology Planning Project of Guangzhou City (202002030236), the Beijing Medical Award Foundation (YXJL-2020-0941-0758), and the Beijing Science and Technology Innovation Medical Development Foundation (KC2022-ZZ-0091-5). Funding sources were not involved in the study design, data collection, analysis and interpretation, writing of the report, or decision to submit the article for publication.
Collapse
Affiliation(s)
- YuHong Huang
- Department of Breast Cancer, Cancer Center, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080 Guangdong, China
| | - Teng Zhu
- Department of Breast Cancer, Cancer Center, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080 Guangdong, China
| | - XiaoLing Zhang
- Department of Radiology, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Wei Li
- Department of Breast Cancer, The First People's Hospital of Foshan, Foshan, Guangdong, China
| | - XingXing Zheng
- Department of Breast Cancer, Cancer Center, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080 Guangdong, China
| | - MinYi Cheng
- Department of Breast Cancer, Cancer Center, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080 Guangdong, China
| | - Fei Ji
- Department of Breast Cancer, Cancer Center, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080 Guangdong, China
| | - LiuLu Zhang
- Department of Breast Cancer, Cancer Center, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080 Guangdong, China
| | - CiQiu Yang
- Department of Breast Cancer, Cancer Center, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080 Guangdong, China
| | - ZhiYong Wu
- Diagnosis and Treatment Center of Breast Diseases, Shantou Central Hospital, Shantou, China
- Corresponding author. Diagnosis and Treatment Center of Breast Diseases, Shantou Central Hospital, Shantou, China
| | - GuoLin Ye
- Department of Breast Cancer, The First People's Hospital of Foshan, Foshan, Guangdong, China
- Corresponding author. Department of Breast Cancer, The First People's Hospital of Foshan, Foshan, 528000, China.
| | - Ying Lin
- Breast Disease Center, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
- Corresponding author. Breast Disease Center, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, 510080, China.
| | - Kun Wang
- Department of Breast Cancer, Cancer Center, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080 Guangdong, China
- Corresponding author. Department of Breast Cancer, Cancer Center, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, 510080, Guangdong, China
| |
Collapse
|
26
|
From patterns to patients: Advances in clinical machine learning for cancer diagnosis, prognosis, and treatment. Cell 2023; 186:1772-1791. [PMID: 36905928 DOI: 10.1016/j.cell.2023.01.035] [Citation(s) in RCA: 76] [Impact Index Per Article: 76.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Revised: 01/10/2023] [Accepted: 01/26/2023] [Indexed: 03/12/2023]
Abstract
Machine learning (ML) is increasingly used in clinical oncology to diagnose cancers, predict patient outcomes, and inform treatment planning. Here, we review recent applications of ML across the clinical oncology workflow. We review how these techniques are applied to medical imaging and to molecular data obtained from liquid and solid tumor biopsies for cancer diagnosis, prognosis, and treatment design. We discuss key considerations in developing ML for the distinct challenges posed by imaging and molecular data. Finally, we examine ML models approved for cancer-related patient usage by regulatory agencies and discuss approaches to improve the clinical usefulness of ML.
Collapse
|
27
|
Prediction of pathologic complete response to neoadjuvant systemic therapy in triple negative breast cancer using deep learning on multiparametric MRI. Sci Rep 2023; 13:1171. [PMID: 36670144 PMCID: PMC9859781 DOI: 10.1038/s41598-023-27518-2] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2022] [Accepted: 01/03/2023] [Indexed: 01/22/2023] Open
Abstract
Triple-negative breast cancer (TNBC) is an aggressive subtype of breast cancer. Neoadjuvant systemic therapy (NAST) followed by surgery are currently standard of care for TNBC with 50-60% of patients achieving pathologic complete response (pCR). We investigated ability of deep learning (DL) on dynamic contrast enhanced (DCE) MRI and diffusion weighted imaging acquired early during NAST to predict TNBC patients' pCR status in the breast. During the development phase using the images of 130 TNBC patients, the DL model achieved areas under the receiver operating characteristic curves (AUCs) of 0.97 ± 0.04 and 0.82 ± 0.10 for the training and the validation, respectively. The model achieved an AUC of 0.86 ± 0.03 when evaluated in the independent testing group of 32 patients. In an additional prospective blinded testing group of 48 patients, the model achieved an AUC of 0.83 ± 0.02. These results demonstrated that DL based on multiparametric MRI can potentially differentiate TNBC patients with pCR or non-pCR in the breast early during NAST.
Collapse
|
28
|
Dagıstanli S, Sonmez S, Bulut N, Kose AM. Evaluation of treatment responses among subgroups of breast cancer patients receiving neoadjuvant chemotherapy. J Cancer Res Ther 2023; 19:S821-S826. [PMID: 38384061 DOI: 10.4103/jcrt.jcrt_1409_22] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Accepted: 02/06/2023] [Indexed: 02/23/2024]
Abstract
BACKGROUND Breast MRIs are helpful for determining treatment plans, responses, and prospective survival analyses. In this retrospective cross-sectional study, we compared the preoperative MRI treatment response to neoadjuvant chemotherapy (NAC) administration with the postoperative pathological response in breast cancer patients. MATERIALS AND METHODS We analyzed data from 108 hospitalized patients receiving NAC between 2020 and 2022. We used MRI to evaluate the treatment response to NAC in patients with locally advanced breast cancers who had not received any prior treatment. We recorded the longest diameter of the primary tumor and the numbers of secondary tumors and axillary lymph nodes. In addition, we examined the correlation between the MRI response rate and pathological specimen results. RESULTS In our subgroup analyses, we found the best pathological response in patients with luminal B (Ki-67 index >14%) breast cancer and positivity for both hormone receptor and HER-2 markers. After comparing the pathological and radiological treatment responses in tumors and lymph nodes, the sensitivities were 90.3% for the pathological assessment and 42.8% for the radiological assessment, while the accuracies were 84.2% for the pathological assessment and 61.1% for the radiological assessment. CONCLUSION Using MRI techniques and sequence intervals and examining the histopathological characteristics of tumors may help increase the accuracy of the pathological complete response.
Collapse
Affiliation(s)
- Sevinc Dagıstanli
- Department of General Surgery, Kanuni Sultan Suleyman Research and Training Hospital, Istanbul, Turkey
| | - Suleyman Sonmez
- Department of Radiology, Kanuni Sultan Suleyman Research and Training Hospital, Istanbul, Turkey
| | - Nilufer Bulut
- Department of Medical Oncology, Basaksehir Cam and Sakura City Hospital, Istanbul, Turkey
| | - Ali Mertcan Kose
- Department of Computer Programming, Vocational School, Istanbul Ticaret University, Istanbul, Turkey
| |
Collapse
|
29
|
Khan N, Adam R, Huang P, Maldjian T, Duong TQ. Deep Learning Prediction of Pathologic Complete Response in Breast Cancer Using MRI and Other Clinical Data: A Systematic Review. Tomography 2022; 8:2784-2795. [PMID: 36412691 PMCID: PMC9680498 DOI: 10.3390/tomography8060232] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2022] [Revised: 11/12/2022] [Accepted: 11/16/2022] [Indexed: 11/24/2022] Open
Abstract
Breast cancer patients who have pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) are more likely to have better clinical outcomes. The ability to predict which patient will respond to NAC early in the treatment course is important because it could help to minimize unnecessary toxic NAC and to modify regimens mid-treatment to achieve better efficacy. Machine learning (ML) is increasingly being used in radiology and medicine because it can identify relationships amongst complex data elements to inform outcomes without the need to specify such relationships a priori. One of the most popular deep learning methods that applies to medical images is the Convolutional Neural Networks (CNN). In contrast to supervised ML, deep learning CNN can operate on the whole images without requiring radiologists to manually contour the tumor on images. Although there have been many review papers on supervised ML prediction of pCR, review papers on deep learning prediction of pCR are sparse. Deep learning CNN could also incorporate multiple image types, clinical data such as demographics and molecular subtypes, as well as data from multiple treatment time points to predict pCR. The goal of this study is to perform a systematic review of deep learning methods that use whole-breast MRI images without annotation or tumor segmentation to predict pCR in breast cancer.
Collapse
|
30
|
A Review of Radiomics in Predicting Therapeutic Response in Colorectal Liver Metastases: From Traditional to Artificial Intelligence Techniques. Healthcare (Basel) 2022; 10:healthcare10102075. [DOI: 10.3390/healthcare10102075] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Revised: 10/12/2022] [Accepted: 10/13/2022] [Indexed: 11/17/2022] Open
Abstract
An early evaluation of colorectal cancer liver metastasis (CRCLM) is crucial in determining treatment options that ultimately affect patient survival rates and outcomes. Radiomics (quantitative imaging features) have recently gained popularity in diagnostic and therapeutic strategies. Despite this, radiomics faces many challenges and limitations. This study sheds light on these limitations by reviewing the studies that used radiomics to predict therapeutic response in CRCLM. Despite radiomics’ potential to enhance clinical decision-making, it lacks standardization. According to the results of this study, the instability of radiomics quantification is caused by changes in CT scan parameters used to obtain CT scans, lesion segmentation methods used for contouring liver metastases, feature extraction methods, and dataset size used for experimentation and validation. Accordingly, the study recommends combining radiomics with deep learning to improve prediction accuracy.
Collapse
|
31
|
Lipkova J, Chen RJ, Chen B, Lu MY, Barbieri M, Shao D, Vaidya AJ, Chen C, Zhuang L, Williamson DFK, Shaban M, Chen TY, Mahmood F. Artificial intelligence for multimodal data integration in oncology. Cancer Cell 2022; 40:1095-1110. [PMID: 36220072 PMCID: PMC10655164 DOI: 10.1016/j.ccell.2022.09.012] [Citation(s) in RCA: 100] [Impact Index Per Article: 50.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/09/2022] [Revised: 07/12/2022] [Accepted: 09/15/2022] [Indexed: 02/07/2023]
Abstract
In oncology, the patient state is characterized by a whole spectrum of modalities, ranging from radiology, histology, and genomics to electronic health records. Current artificial intelligence (AI) models operate mainly in the realm of a single modality, neglecting the broader clinical context, which inevitably diminishes their potential. Integration of different data modalities provides opportunities to increase robustness and accuracy of diagnostic and prognostic models, bringing AI closer to clinical practice. AI models are also capable of discovering novel patterns within and across modalities suitable for explaining differences in patient outcomes or treatment resistance. The insights gleaned from such models can guide exploration studies and contribute to the discovery of novel biomarkers and therapeutic targets. To support these advances, here we present a synopsis of AI methods and strategies for multimodal data fusion and association discovery. We outline approaches for AI interpretability and directions for AI-driven exploration through multimodal data interconnections. We examine challenges in clinical adoption and discuss emerging solutions.
Collapse
Affiliation(s)
- Jana Lipkova
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Richard J Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Bowen Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Department of Computer Science, Harvard University, Cambridge, MA, USA
| | - Ming Y Lu
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology (MIT), Cambridge, MA, USA
| | - Matteo Barbieri
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Daniel Shao
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Harvard-MIT Health Sciences and Technology (HST), Cambridge, MA, USA
| | - Anurag J Vaidya
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Harvard-MIT Health Sciences and Technology (HST), Cambridge, MA, USA
| | - Chengkuan Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Luoting Zhuang
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
| | - Drew F K Williamson
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Muhammad Shaban
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Tiffany Y Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Faisal Mahmood
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Harvard Data Science Initiative, Harvard University, Cambridge, MA, USA.
| |
Collapse
|
32
|
Multimodal Prediction of Five-Year Breast Cancer Recurrence in Women Who Receive Neoadjuvant Chemotherapy. Cancers (Basel) 2022; 14:cancers14163848. [PMID: 36010844 PMCID: PMC9405765 DOI: 10.3390/cancers14163848] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2022] [Revised: 07/29/2022] [Accepted: 08/04/2022] [Indexed: 11/17/2022] Open
Abstract
In current clinical practice, it is difficult to predict whether a patient receiving neoadjuvant chemotherapy (NAC) for breast cancer is likely to encounter recurrence after treatment and have the cancer recur locally in the breast or in other areas of the body. We explore the use of clinical history, immunohistochemical markers, and multiparametric magnetic resonance imaging (DCE, ADC, Dixon) to predict the risk of post-treatment recurrence within five years. We performed a retrospective study on a cohort of 1738 patients from Institut Curie and analyzed the data using classical machine learning, image processing, and deep learning. Our results demonstrate the ability to predict recurrence prior to NAC treatment initiation using each modality alone, and the possible improvement achieved by combining the modalities. When evaluated on holdout data, the multimodal model achieved an AUC of 0.75 (CI: 0.70, 0.80) and 0.57 specificity at 0.90 sensitivity. We then stratified the data based on known prognostic biomarkers. We found that our models can provide accurate recurrence predictions (AUC > 0.89) for specific groups of women under 50 years old with poor prognoses. A version of our method won second place at the BMMR2 Challenge, with a very small margin from being first, and was a standout from the other challenge entries.
Collapse
|
33
|
Kim SY, Cho N. Breast Magnetic Resonance Imaging for Patients With Newly Diagnosed Breast Cancer: A Review. J Breast Cancer 2022; 25:263-277. [PMID: 36031752 PMCID: PMC9411024 DOI: 10.4048/jbc.2022.25.e35] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Revised: 07/13/2022] [Accepted: 07/25/2022] [Indexed: 11/30/2022] Open
Abstract
Despite the high sensitivity and widespread use of preoperative magnetic resonance imaging (MRI), the American Cancer Society and the National Comprehensive Cancer Network guidelines do not recommend the routine use of preoperative MRI owing to the conflicting results and lack of clear benefit to the surgical outcome (reoperation and mastectomy) and long-term clinical outcomes (local recurrence and metachronous contralateral breast cancer). Preoperative MRI detects additional cancers that are occult at mammography and ultrasound but increases the rate of mastectomy. Concerns about overdiagnosis and overtreatment of preoperative MRI might be mitigated by adjusting the confounding factors when conducting studies, using the state-of-the-art image-guided biopsy technique, applying the radiologists’ cumulative experiences in interpreting MRI findings, and performing multiple lumpectomies in patients with multicentric cancer. Among the various imaging methods, dynamic contrast-enhanced MRI has the highest accuracy in predicting pathologic complete response after neoadjuvant chemotherapy. Prospective trials aimed at applying the MRI information to the de-escalation of surgical or radiation treatments are underway. In this review, current studies on the clinical outcomes of preoperative breast MRI are updated, and circumstances in which MRI may be useful for surgical planning are discussed.
Collapse
Affiliation(s)
- Soo-Yeon Kim
- Department of Radiology, Seoul National University Hospital, Seoul, Korea.,Department of Radiology, Seoul National University College of Medicine, Seoul, Korea.,Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea
| | - Nariya Cho
- Department of Radiology, Seoul National University Hospital, Seoul, Korea.,Department of Radiology, Seoul National University College of Medicine, Seoul, Korea.,Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea.
| |
Collapse
|
34
|
Liang X, Yu X, Gao T. Machine learning with magnetic resonance imaging for prediction of response to neoadjuvant chemotherapy in breast cancer: A systematic review and meta-analysis. Eur J Radiol 2022; 150:110247. [DOI: 10.1016/j.ejrad.2022.110247] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2021] [Revised: 02/14/2022] [Accepted: 03/07/2022] [Indexed: 12/18/2022]
|
35
|
Zhang X, Zhang Y, Zhang G, Qiu X, Tan W, Yin X, Liao L. Deep Learning With Radiomics for Disease Diagnosis and Treatment: Challenges and Potential. Front Oncol 2022; 12:773840. [PMID: 35251962 PMCID: PMC8891653 DOI: 10.3389/fonc.2022.773840] [Citation(s) in RCA: 37] [Impact Index Per Article: 18.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2021] [Accepted: 01/17/2022] [Indexed: 12/12/2022] Open
Abstract
The high-throughput extraction of quantitative imaging features from medical images for the purpose of radiomic analysis, i.e., radiomics in a broad sense, is a rapidly developing and emerging research field that has been attracting increasing interest, particularly in multimodality and multi-omics studies. In this context, the quantitative analysis of multidimensional data plays an essential role in assessing the spatio-temporal characteristics of different tissues and organs and their microenvironment. Herein, recent developments in this method, including manually defined features, data acquisition and preprocessing, lesion segmentation, feature extraction, feature selection and dimension reduction, statistical analysis, and model construction, are reviewed. In addition, deep learning-based techniques for automatic segmentation and radiomic analysis are being analyzed to address limitations such as rigorous workflow, manual/semi-automatic lesion annotation, and inadequate feature criteria, and multicenter validation. Furthermore, a summary of the current state-of-the-art applications of this technology in disease diagnosis, treatment response, and prognosis prediction from the perspective of radiology images, multimodality images, histopathology images, and three-dimensional dose distribution data, particularly in oncology, is presented. The potential and value of radiomics in diagnostic and therapeutic strategies are also further analyzed, and for the first time, the advances and challenges associated with dosiomics in radiotherapy are summarized, highlighting the latest progress in radiomics. Finally, a robust framework for radiomic analysis is presented and challenges and recommendations for future development are discussed, including but not limited to the factors that affect model stability (medical big data and multitype data and expert knowledge in medical), limitations of data-driven processes (reproducibility and interpretability of studies, different treatment alternatives for various institutions, and prospective researches and clinical trials), and thoughts on future directions (the capability to achieve clinical applications and open platform for radiomics analysis).
Collapse
Affiliation(s)
- Xingping Zhang
- Institute of Advanced Cyberspace Technology, Guangzhou University, Guangzhou, China
- Department of New Networks, Peng Cheng Laboratory, Shenzhen, China
| | - Yanchun Zhang
- Institute of Advanced Cyberspace Technology, Guangzhou University, Guangzhou, China
- Department of New Networks, Peng Cheng Laboratory, Shenzhen, China
| | - Guijuan Zhang
- Department of Respiratory Medicine, First Affiliated Hospital of Gannan Medical University, Ganzhou, China
| | - Xingting Qiu
- Department of Radiology, First Affiliated Hospital of Gannan Medical University, Ganzhou, China
| | - Wenjun Tan
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Shenyang, China
| | - Xiaoxia Yin
- Institute of Advanced Cyberspace Technology, Guangzhou University, Guangzhou, China
| | - Liefa Liao
- School of Information Engineering, Jiangxi University of Science and Technology, Ganzhou, China
| |
Collapse
|