1
|
Tong Y, Hu Z, Wang H, Huang J, Zhan Y, Chai W, Deng Y, Yuan Y, Shen K, Wang Y, Chen X, Yu J. Anti-HER2 therapy response assessment for guiding treatment (de-)escalation in early HER2-positive breast cancer using a novel deep learning radiomics model. Eur Radiol 2024; 34:5477-5486. [PMID: 38329503 PMCID: PMC11255056 DOI: 10.1007/s00330-024-10609-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2023] [Revised: 11/24/2023] [Accepted: 01/01/2024] [Indexed: 02/09/2024]
Abstract
OBJECTIVES Anti-HER2 targeted therapy significantly reduces risk of relapse in HER2 + breast cancer. New measures are needed for a precise risk stratification to guide (de-)escalation of anti-HER2 strategy. METHODS A total of 726 HER2 + cases who received no/single/dual anti-HER2 targeted therapies were split into three respective cohorts. A deep learning model (DeepTEPP) based on preoperative breast magnetic resonance (MR) was developed. Patients were scored and categorized into low-, moderate-, and high-risk groups. Recurrence-free survival (RFS) was compared in patients with different risk groups according to the anti-HER2 treatment they received, to validate the value of DeepTEPP in predicting treatment efficacy and guiding anti-HER2 strategy. RESULTS DeepTEPP was capable of risk stratification and guiding anti-HER2 treatment strategy: DeepTEPP-Low patients (60.5%) did not derive significant RFS benefit from trastuzumab (p = 0.144), proposing an anti-HER2 de-escalation. DeepTEPP-Moderate patients (19.8%) significantly benefited from trastuzumab (p = 0.048), but did not obtain additional improvements from pertuzumab (p = 0.125). DeepTEPP-High patients (19.7%) significantly benefited from dual HER2 blockade (p = 0.045), suggesting an anti-HER2 escalation. CONCLUSIONS DeepTEPP represents a pioneering MR-based deep learning model that enables the non-invasive prediction of adjuvant anti-HER2 effectiveness, thereby providing valuable guidance for anti-HER2 (de-)escalation strategies. DeepTEPP provides an important reference for choosing the appropriate individualized treatment in HER2 + breast cancer patients, warranting prospective validation. CLINICAL RELEVANCE STATEMENT We built an MR-based deep learning model DeepTEPP, which enables the non-invasive prediction of adjuvant anti-HER2 effectiveness, thus guiding anti-HER2 (de-)escalation strategies in early HER2-positive breast cancer patients. KEY POINTS • DeepTEPP is able to predict anti-HER2 effectiveness and to guide treatment (de-)escalation. • DeepTEPP demonstrated an impressive prognostic efficacy for recurrence-free survival and overall survival. • To our knowledge, this is one of the very few, also the largest study to test the efficacy of a deep learning model extracted from breast MR images on HER2-positive breast cancer survival and anti-HER2 therapy effectiveness prediction.
Collapse
Affiliation(s)
- Yiwei Tong
- Department of General Surgery, Comprehensive Breast Health Center, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, 197 Ruijin Er Road, Shanghai, 200025, China
| | - Zhaoyu Hu
- School of Information Science and Technology, Fudan University, No. 220, Handan Road, Yangpu District, Shanghai, 200433, China
| | - Haoyu Wang
- Department of General Surgery, Comprehensive Breast Health Center, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, 197 Ruijin Er Road, Shanghai, 200025, China
| | - Jiahui Huang
- Department of General Surgery, Comprehensive Breast Health Center, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, 197 Ruijin Er Road, Shanghai, 200025, China
| | - Ying Zhan
- Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China
| | - Weimin Chai
- Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China
| | - Yinhui Deng
- School of Information Science and Technology, Fudan University, No. 220, Handan Road, Yangpu District, Shanghai, 200433, China
| | - Ying Yuan
- Department of Radiology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China
| | - Kunwei Shen
- Department of General Surgery, Comprehensive Breast Health Center, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, 197 Ruijin Er Road, Shanghai, 200025, China
| | - Yuanyuan Wang
- School of Information Science and Technology, Fudan University, No. 220, Handan Road, Yangpu District, Shanghai, 200433, China
| | - Xiaosong Chen
- Department of General Surgery, Comprehensive Breast Health Center, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, 197 Ruijin Er Road, Shanghai, 200025, China.
| | - Jinhua Yu
- School of Information Science and Technology, Fudan University, No. 220, Handan Road, Yangpu District, Shanghai, 200433, China.
| |
Collapse
|
2
|
Zou Y, Xue M, Hossain MI, Zhu Q. Ultrasound and diffuse optical tomography-transformer model for assessing pathological complete response to neoadjuvant chemotherapy in breast cancer. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:076007. [PMID: 39050779 PMCID: PMC11268382 DOI: 10.1117/1.jbo.29.7.076007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/23/2024] [Revised: 06/28/2024] [Accepted: 07/01/2024] [Indexed: 07/27/2024]
Abstract
Significance We evaluate the efficiency of integrating ultrasound (US) and diffuse optical tomography (DOT) images for predicting pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) in breast cancer patients. The ultrasound-diffuse optical tomography (USDOT)-Transformer model represents a significant step toward accurate prediction of pCR, which is critical for personalized treatment planning. Aim We aim to develop and assess the performance of the USDOT-Transformer model, which combines US and DOT images with tumor receptor biomarkers to predict the pCR of breast cancer patients under NAC. Approach We developed the USDOT-Transformer model using a dual-input transformer to process co-registered US and DOT images along with tumor receptor biomarkers. Our dataset comprised imaging data from 60 patients at multiple time points during their chemotherapy treatment. We used fivefold cross-validation to assess the model's performance, comparing its results against a single modality of US or DOT. Results The USDOT-Transformer model demonstrated excellent predictive performance, with a mean area under the receiving characteristic curve of 0.96 (95%CI: 0.93 to 0.99) across the fivefold cross-validation. The integration of US and DOT images significantly enhanced the model's ability to predict pCR, outperforming models that relied on a single imaging modality (0.87 for US and 0.82 for DOT). This performance indicates the potential of advanced deep learning techniques and multimodal imaging data for improving the accuracy (ACC) of pCR prediction. Conclusion The USDOT-Transformer model offers a promising non-invasive approach for predicting pCR to NAC in breast cancer patients. By leveraging the structural and functional information from US and DOT images, the model offers a faster and more reliable tool for personalized treatment planning. Future work will focus on expanding the dataset and refining the model to further improve its accuracy and generalizability.
Collapse
Affiliation(s)
- Yun Zou
- Washington University in St. Louis, Department of Biomedical Engineering, St. Louis, Missouri, United States
| | - Minghao Xue
- Washington University in St. Louis, Department of Biomedical Engineering, St. Louis, Missouri, United States
| | - Md Iqbal Hossain
- Washington University in St. Louis, Imaging Science, St. Louis, Missouri, United States
| | - Quing Zhu
- Washington University in St. Louis, Department of Biomedical Engineering, St. Louis, Missouri, United States
- Washington University School of Medicine, Department of Radiology, St. Louis, Missouri, United States
| |
Collapse
|
3
|
Mathur A, Arya N, Pasupa K, Saha S, Roy Dey S, Saha S. Breast cancer prognosis through the use of multi-modal classifiers: current state of the art and the way forward. Brief Funct Genomics 2024:elae015. [PMID: 38688724 DOI: 10.1093/bfgp/elae015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Revised: 03/01/2024] [Accepted: 04/09/2024] [Indexed: 05/02/2024] Open
Abstract
We present a survey of the current state-of-the-art in breast cancer detection and prognosis. We analyze the evolution of Artificial Intelligence-based approaches from using just uni-modal information to multi-modality for detection and how such paradigm shift facilitates the efficacy of detection, consistent with clinical observations. We conclude that interpretable AI-based predictions and ability to handle class imbalance should be considered priority.
Collapse
Affiliation(s)
- Archana Mathur
- Department of Information Science and Engineering, Nitte Meenakshi Institute of Technology, Yelahanka, 560064, Karnataka, India
| | - Nikhilanand Arya
- School of Computer Engineering, Kalinga Institute of Industrial Technology, Deemed to be University, Bhubaneshwar, 751024, Odisha, India
| | - Kitsuchart Pasupa
- School of Information Technology, King Mongkut's Institute of Technology Ladkrabang, 1 Soi Chalongkrung 1, 10520, Bangkok, Thailand
| | - Sriparna Saha
- Computer Science and Engineering, Indian Institute of Technology Patna, Bihta, 801106, Bihar, India
| | - Sudeepa Roy Dey
- Department of Computer Science and Engineering, PES University, Hosur Road, 560100, Karnataka, India
| | - Snehanshu Saha
- CSIS and APPCAIR, BITS Pilani K.K Birla Goa Campus, Goa, 403726, Goa, India
- Div of AI Research, HappyMonk AI, Bangalore, 560078, Karnataka, India
| |
Collapse
|
4
|
Lo Gullo R, Brunekreef J, Marcus E, Han LK, Eskreis-Winkler S, Thakur SB, Mann R, Groot Lipman K, Teuwen J, Pinker K. AI Applications to Breast MRI: Today and Tomorrow. J Magn Reson Imaging 2024. [PMID: 38581127 DOI: 10.1002/jmri.29358] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Revised: 03/07/2024] [Accepted: 03/09/2024] [Indexed: 04/08/2024] Open
Abstract
In breast imaging, there is an unrelenting increase in the demand for breast imaging services, partly explained by continuous expanding imaging indications in breast diagnosis and treatment. As the human workforce providing these services is not growing at the same rate, the implementation of artificial intelligence (AI) in breast imaging has gained significant momentum to maximize workflow efficiency and increase productivity while concurrently improving diagnostic accuracy and patient outcomes. Thus far, the implementation of AI in breast imaging is at the most advanced stage with mammography and digital breast tomosynthesis techniques, followed by ultrasound, whereas the implementation of AI in breast magnetic resonance imaging (MRI) is not moving along as rapidly due to the complexity of MRI examinations and fewer available dataset. Nevertheless, there is persisting interest in AI-enhanced breast MRI applications, even as the use of and indications of breast MRI continue to expand. This review presents an overview of the basic concepts of AI imaging analysis and subsequently reviews the use cases for AI-enhanced MRI interpretation, that is, breast MRI triaging and lesion detection, lesion classification, prediction of treatment response, risk assessment, and image quality. Finally, it provides an outlook on the barriers and facilitators for the adoption of AI in breast MRI. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 6.
Collapse
Affiliation(s)
- Roberto Lo Gullo
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, New York, USA
| | - Joren Brunekreef
- AI for Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
| | - Eric Marcus
- AI for Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
| | - Lynn K Han
- Weill Cornell Medical College, New York-Presbyterian Hospital, New York City, New York, USA
| | - Sarah Eskreis-Winkler
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, New York, USA
| | - Sunitha B Thakur
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, New York, USA
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York City, New York, USA
| | - Ritse Mann
- AI for Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Kevin Groot Lipman
- AI for Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Jonas Teuwen
- AI for Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Katja Pinker
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, New York, USA
| |
Collapse
|
5
|
Comes MC, Fanizzi A, Bove S, Didonna V, Diotiaiuti S, Fadda F, La Forgia D, Giotta F, Latorre A, Nardone A, Palmiotti G, Ressa CM, Rinaldi L, Rizzo A, Talienti T, Tamborra P, Zito A, Lorusso V, Massafra R. Explainable 3D CNN based on baseline breast DCE-MRI to give an early prediction of pathological complete response to neoadjuvant chemotherapy. Comput Biol Med 2024; 172:108132. [PMID: 38508058 DOI: 10.1016/j.compbiomed.2024.108132] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Revised: 01/29/2024] [Accepted: 02/12/2024] [Indexed: 03/22/2024]
Abstract
BACKGROUND So far, baseline Dynamic Contrast-Enhanced Magnetic Resonance Imaging (DCE-MRI) has played a key role for the application of sophisticated artificial intelligence-based models using Convolutional Neural Networks (CNNs) to extract quantitative imaging information as earlier indicators of pathological Complete Response (pCR) achievement in breast cancer patients treated with neoadjuvant chemotherapy (NAC). However, these models did not exploit the DCE-MRI exams in their full geometry as 3D volume but analysed only few individual slices independently, thus neglecting the depth information. METHOD This study aimed to develop an explainable 3D CNN, which fulfilled the task of pCR prediction before the beginning of NAC, by leveraging the 3D information of post-contrast baseline breast DCE-MRI exams. Specifically, for each patient, the network took in input a 3D sequence containing the tumor region, which was previously automatically identified along the DCE-MRI exam. A visual explanation of the decision-making process of the network was also provided. RESULTS To the best of our knowledge, our proposal is competitive than other models in the field, which made use of imaging data alone, reaching a median AUC value of 81.8%, 95%CI [75.3%; 88.3%], a median accuracy value of 78.7%, 95%CI [74.8%; 82.5%], a median sensitivity value of 69.8%, 95%CI [59.6%; 79.9%] and a median specificity value of 83.3%, 95%CI [82.6%; 84.0%], respectively. The median and CIs were computed according to a 10-fold cross-validation scheme for 5 rounds. CONCLUSION Finally, this proposal holds high potential to support clinicians on non-invasively early pursuing or changing patient-centric NAC pathways.
Collapse
Affiliation(s)
- Maria Colomba Comes
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Annarita Fanizzi
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy.
| | - Samantha Bove
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy.
| | - Vittorio Didonna
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Sergio Diotiaiuti
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Federico Fadda
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Daniele La Forgia
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Francesco Giotta
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Agnese Latorre
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Annalisa Nardone
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Gennaro Palmiotti
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Cosmo Maurizio Ressa
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Lucia Rinaldi
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Alessandro Rizzo
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Tiziana Talienti
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Pasquale Tamborra
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Alfredo Zito
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Vito Lorusso
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Raffaella Massafra
- I.R.C.C.S. Istituto Tumori "Giovanni Paolo II", Viale Orazio Flacco 65, 70124, Bari, Italy
| |
Collapse
|
6
|
Seth I, Lim B, Joseph K, Gracias D, Xie Y, Ross RJ, Rozen WM. Use of artificial intelligence in breast surgery: a narrative review. Gland Surg 2024; 13:395-411. [PMID: 38601286 PMCID: PMC11002485 DOI: 10.21037/gs-23-414] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2023] [Accepted: 02/21/2024] [Indexed: 04/12/2024]
Abstract
Background and Objective We have witnessed tremendous advances in artificial intelligence (AI) technologies. Breast surgery, a subspecialty of general surgery, has notably benefited from AI technologies. This review aims to evaluate how AI has been integrated into breast surgery practices, to assess its effectiveness in improving surgical outcomes and operational efficiency, and to identify potential areas for future research and application. Methods Two authors independently conducted a comprehensive search of PubMed, Google Scholar, EMBASE, and Cochrane CENTRAL databases from January 1, 1950, to September 4, 2023, employing keywords pertinent to AI in conjunction with breast surgery or cancer. The search focused on English language publications, where relevance was determined through meticulous screening of titles, abstracts, and full-texts, followed by an additional review of references within these articles. The review covered a range of studies illustrating the applications of AI in breast surgery encompassing lesion diagnosis to postoperative follow-up. Publications focusing specifically on breast reconstruction were excluded. Key Content and Findings AI models have preoperative, intraoperative, and postoperative applications in the field of breast surgery. Using breast imaging scans and patient data, AI models have been designed to predict the risk of breast cancer and determine the need for breast cancer surgery. In addition, using breast imaging scans and histopathological slides, models were used for detecting, classifying, segmenting, grading, and staging breast tumors. Preoperative applications included patient education and the display of expected aesthetic outcomes. Models were also designed to provide intraoperative assistance for precise tumor resection and margin status assessment. As well, AI was used to predict postoperative complications, survival, and cancer recurrence. Conclusions Extra research is required to move AI models from the experimental stage to actual implementation in healthcare. With the rapid evolution of AI, further applications are expected in the coming years including direct performance of breast surgery. Breast surgeons should be updated with the advances in AI applications in breast surgery to provide the best care for their patients.
Collapse
Affiliation(s)
- Ishith Seth
- Department of Plastic Surgery, Peninsula Health, Melbourne, Victoria, Australia
- Central Clinical School at Monash University, The Alfred Centre, Melbourne, Victoria, Australia
| | - Bryan Lim
- Department of Plastic Surgery, Peninsula Health, Melbourne, Victoria, Australia
- Central Clinical School at Monash University, The Alfred Centre, Melbourne, Victoria, Australia
| | - Konrad Joseph
- Department of Surgery, Port Macquarie Base Hospital, New South Wales, Australia
| | - Dylan Gracias
- Department of Surgery, Townsville Hospital, Queensland, Australia
| | - Yi Xie
- Department of Plastic Surgery, Peninsula Health, Melbourne, Victoria, Australia
| | - Richard J. Ross
- Department of Plastic Surgery, Peninsula Health, Melbourne, Victoria, Australia
- Central Clinical School at Monash University, The Alfred Centre, Melbourne, Victoria, Australia
| | - Warren M. Rozen
- Department of Plastic Surgery, Peninsula Health, Melbourne, Victoria, Australia
- Central Clinical School at Monash University, The Alfred Centre, Melbourne, Victoria, Australia
| |
Collapse
|
7
|
Lo Gullo R, Marcus E, Huayanay J, Eskreis-Winkler S, Thakur S, Teuwen J, Pinker K. Artificial Intelligence-Enhanced Breast MRI: Applications in Breast Cancer Primary Treatment Response Assessment and Prediction. Invest Radiol 2024; 59:230-242. [PMID: 37493391 PMCID: PMC10818006 DOI: 10.1097/rli.0000000000001010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/27/2023]
Abstract
ABSTRACT Primary systemic therapy (PST) is the treatment of choice in patients with locally advanced breast cancer and is nowadays also often used in patients with early-stage breast cancer. Although imaging remains pivotal to assess response to PST accurately, the use of imaging to predict response to PST has the potential to not only better prognostication but also allow the de-escalation or omission of potentially toxic treatment with undesirable adverse effects, the accelerated implementation of new targeted therapies, and the mitigation of surgical delays in selected patients. In response to the limited ability of radiologists to predict response to PST via qualitative, subjective assessments of tumors on magnetic resonance imaging (MRI), artificial intelligence-enhanced MRI with classical machine learning, and in more recent times, deep learning, have been used with promising results to predict response, both before the start of PST and in the early stages of treatment. This review provides an overview of the current applications of artificial intelligence to MRI in assessing and predicting response to PST, and discusses the challenges and limitations of their clinical implementation.
Collapse
Affiliation(s)
- Roberto Lo Gullo
- Department of Radiology, Breast Imaging Service, Memorial Sloan Kettering Cancer Center, 300 E 66 Street, New York, NY 10065, USA
| | - Eric Marcus
- AI for Oncology, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, the Netherlands
| | - Jorge Huayanay
- Department of Radiology, Breast Imaging Service, Memorial Sloan Kettering Cancer Center, 300 E 66 Street, New York, NY 10065, USA
- Department of Radiology, National Institute of Neoplastic Diseases, Lima, Peru
| | - Sarah Eskreis-Winkler
- Department of Radiology, Breast Imaging Service, Memorial Sloan Kettering Cancer Center, 300 E 66 Street, New York, NY 10065, USA
| | - Sunitha Thakur
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, New York, USA
| | - Jonas Teuwen
- Department of Radiology, Breast Imaging Service, Memorial Sloan Kettering Cancer Center, 300 E 66 Street, New York, NY 10065, USA
- Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands
- AI for Oncology, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX Amsterdam, the Netherlands
| | - Katja Pinker
- Department of Radiology, Breast Imaging Service, Memorial Sloan Kettering Cancer Center, 300 E 66 Street, New York, NY 10065, USA
| |
Collapse
|
8
|
Cobanaj M, Corti C, Dee EC, McCullum L, Boldrini L, Schlam I, Tolaney SM, Celi LA, Curigliano G, Criscitiello C. Advancing equitable and personalized cancer care: Novel applications and priorities of artificial intelligence for fairness and inclusivity in the patient care workflow. Eur J Cancer 2024; 198:113504. [PMID: 38141549 DOI: 10.1016/j.ejca.2023.113504] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2023] [Accepted: 12/13/2023] [Indexed: 12/25/2023]
Abstract
Patient care workflows are highly multimodal and intertwined: the intersection of data outputs provided from different disciplines and in different formats remains one of the main challenges of modern oncology. Artificial Intelligence (AI) has the potential to revolutionize the current clinical practice of oncology owing to advancements in digitalization, database expansion, computational technologies, and algorithmic innovations that facilitate discernment of complex relationships in multimodal data. Within oncology, radiation therapy (RT) represents an increasingly complex working procedure, involving many labor-intensive and operator-dependent tasks. In this context, AI has gained momentum as a powerful tool to standardize treatment performance and reduce inter-observer variability in a time-efficient manner. This review explores the hurdles associated with the development, implementation, and maintenance of AI platforms and highlights current measures in place to address them. In examining AI's role in oncology workflows, we underscore that a thorough and critical consideration of these challenges is the only way to ensure equitable and unbiased care delivery, ultimately serving patients' survival and quality of life.
Collapse
Affiliation(s)
- Marisa Cobanaj
- National Center for Radiation Research in Oncology, OncoRay, Helmholtz-Zentrum Dresden-Rossendorf, Dresden, Germany
| | - Chiara Corti
- Breast Oncology Program, Dana-Farber Brigham Cancer Center, Boston, MA, USA; Harvard Medical School, Boston, MA, USA; Division of New Drugs and Early Drug Development for Innovative Therapies, European Institute of Oncology, IRCCS, Milan, Italy; Department of Oncology and Hematology-Oncology (DIPO), University of Milan, Milan, Italy.
| | - Edward C Dee
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Lucas McCullum
- Department of Radiation Oncology, MD Anderson Cancer Center, Houston, TX, USA
| | - Laura Boldrini
- Division of New Drugs and Early Drug Development for Innovative Therapies, European Institute of Oncology, IRCCS, Milan, Italy; Department of Oncology and Hematology-Oncology (DIPO), University of Milan, Milan, Italy
| | - Ilana Schlam
- Department of Hematology and Oncology, Tufts Medical Center, Boston, MA, USA; Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Sara M Tolaney
- Breast Oncology Program, Dana-Farber Brigham Cancer Center, Boston, MA, USA; Harvard Medical School, Boston, MA, USA; Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Leo A Celi
- Department of Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA; Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA, USA; Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Giuseppe Curigliano
- Division of New Drugs and Early Drug Development for Innovative Therapies, European Institute of Oncology, IRCCS, Milan, Italy; Department of Oncology and Hematology-Oncology (DIPO), University of Milan, Milan, Italy
| | - Carmen Criscitiello
- Division of New Drugs and Early Drug Development for Innovative Therapies, European Institute of Oncology, IRCCS, Milan, Italy; Department of Oncology and Hematology-Oncology (DIPO), University of Milan, Milan, Italy
| |
Collapse
|
9
|
Cong C, Li X, Zhang C, Zhang J, Sun K, Liu L, Ambale-Venkatesh B, Chen X, Wang Y. MRI-Based Breast Cancer Classification and Localization by Multiparametric Feature Extraction and Combination Using Deep Learning. J Magn Reson Imaging 2024; 59:148-161. [PMID: 37013422 DOI: 10.1002/jmri.28713] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Revised: 03/16/2023] [Accepted: 03/16/2023] [Indexed: 04/05/2023] Open
Abstract
BACKGROUND Deep learning (DL) have been reported feasible in breast MRI. However, the effectiveness of DL method in mpMRI combinations for breast cancer detection has not been well investigated. PURPOSE To implement a DL method for breast cancer classification and detection using feature extraction and combination from multiple sequences. STUDY TYPE Retrospective. POPULATION A total of 569 local cases as internal cohort (50.2 ± 11.2 years; 100% female), divided among training (218), validation (73) and testing (278); 125 cases from a public dataset as the external cohort (53.6 ± 11.5 years; 100% female). FIELD STRENGTH/SEQUENCE T1-weighted imaging and dynamic contrast-enhanced MRI (DCE-MRI) with gradient echo sequences, T2-weighted imaging (T2WI) with spin-echo sequences, diffusion-weighted imaging with single-shot echo-planar sequence and at 1.5-T. ASSESSMENT A convolutional neural network and long short-term memory cascaded network was implemented for lesion classification with histopathology as the ground truth for malignant and benign categories and contralateral breasts as healthy category in internal/external cohorts. BI-RADS categories were assessed by three independent radiologists as comparison, and class activation map was employed for lesion localization in internal cohort. The classification and localization performances were assessed with DCE-MRI and non-DCE sequences, respectively. STATISTICAL TESTS Sensitivity, specificity, area under the curve (AUC), DeLong test, and Cohen's kappa for lesion classification. Sensitivity and mean squared error for localization. A P-value <0.05 was considered statistically significant. RESULTS With the optimized mpMRI combinations, the lesion classification achieved an AUC = 0.98/0.91, sensitivity = 0.96/0.83 in the internal/external cohorts, respectively. Without DCE-MRI, the DL-based method was superior to radiologists' readings (AUC 0.96 vs. 0.90). The lesion localization achieved sensitivities of 0.97/0.93 with DCE-MRI/T2WI alone, respectively. DATA CONCLUSION The DL method achieved high accuracy for lesion detection in the internal/external cohorts. The classification performance with a contrast agent-free combination is comparable to DCE-MRI alone and the radiologists' reading in AUC and sensitivity. EVIDENCE LEVEL 3. TECHNICAL EFFICACY Stage 2.
Collapse
Affiliation(s)
- Chao Cong
- Department of Radiology, Daping Hospital, Army Medical University, Chongqing, China
- School of Electrical and Electronic Engineering, Chongqing University of Technology, Chongqing, China
- Department of Nuclear Medicine, Daping Hospital, Army Medical University, Chongqing, China
| | - Xiaoguang Li
- Department of Radiology, Daping Hospital, Army Medical University, Chongqing, China
| | - Chunlai Zhang
- Department of Radiology, Daping Hospital, Army Medical University, Chongqing, China
| | - Jing Zhang
- Department of Radiology, Daping Hospital, Army Medical University, Chongqing, China
| | - Kaixiang Sun
- School of Electrical and Electronic Engineering, Chongqing University of Technology, Chongqing, China
| | - Lianluyi Liu
- School of Electrical and Electronic Engineering, Chongqing University of Technology, Chongqing, China
| | | | - Xiao Chen
- Department of Nuclear Medicine, Daping Hospital, Army Medical University, Chongqing, China
| | - Yi Wang
- Department of Nuclear Medicine, Daping Hospital, Army Medical University, Chongqing, China
| |
Collapse
|
10
|
Ahn JS, Shin S, Yang SA, Park EK, Kim KH, Cho SI, Ock CY, Kim S. Artificial Intelligence in Breast Cancer Diagnosis and Personalized Medicine. J Breast Cancer 2023; 26:405-435. [PMID: 37926067 PMCID: PMC10625863 DOI: 10.4048/jbc.2023.26.e45] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 09/25/2023] [Accepted: 10/06/2023] [Indexed: 11/07/2023] Open
Abstract
Breast cancer is a significant cause of cancer-related mortality in women worldwide. Early and precise diagnosis is crucial, and clinical outcomes can be markedly enhanced. The rise of artificial intelligence (AI) has ushered in a new era, notably in image analysis, paving the way for major advancements in breast cancer diagnosis and individualized treatment regimens. In the diagnostic workflow for patients with breast cancer, the role of AI encompasses screening, diagnosis, staging, biomarker evaluation, prognostication, and therapeutic response prediction. Although its potential is immense, its complete integration into clinical practice is challenging. Particularly, these challenges include the imperatives for extensive clinical validation, model generalizability, navigating the "black-box" conundrum, and pragmatic considerations of embedding AI into everyday clinical environments. In this review, we comprehensively explored the diverse applications of AI in breast cancer care, underlining its transformative promise and existing impediments. In radiology, we specifically address AI in mammography, tomosynthesis, risk prediction models, and supplementary imaging methods, including magnetic resonance imaging and ultrasound. In pathology, our focus is on AI applications for pathologic diagnosis, evaluation of biomarkers, and predictions related to genetic alterations, treatment response, and prognosis in the context of breast cancer diagnosis and treatment. Our discussion underscores the transformative potential of AI in breast cancer management and emphasizes the importance of focused research to realize the full spectrum of benefits of AI in patient care.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | - Seokhwi Kim
- Department of Pathology, Ajou University School of Medicine, Suwon, Korea
- Department of Biomedical Sciences, Ajou University Graduate School of Medicine, Suwon, Korea.
| |
Collapse
|
11
|
Huang JX, Shi J, Ding SS, Zhang HL, Wang XY, Lin SY, Xu YF, Wei MJ, Liu LZ, Pei XQ. Deep Learning Model Based on Dual-Modal Ultrasound and Molecular Data for Predicting Response to Neoadjuvant Chemotherapy in Breast Cancer. Acad Radiol 2023; 30 Suppl 2:S50-S61. [PMID: 37270368 DOI: 10.1016/j.acra.2023.03.036] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 03/24/2023] [Accepted: 03/25/2023] [Indexed: 06/05/2023]
Abstract
RATIONALE AND OBJECTIVES To carry out radiomics analysis/deep convolutional neural network (CNN) based on B-mode ultrasound (BUS) and shear wave elastography (SWE) to predict response to neoadjuvant chemotherapy (NAC) in breast cancer patients. MATERIALS AND METHODS In this prospective study, 255 breast cancer patients who received NAC between September 2016 and December 2021 were included. Radiomics models were designed using a support vector machine classifier based on US images obtained before treatment, including BUS and SWE. And CNN models also were developed using ResNet architecture. The final predictive model was developed by combining the dual-modal US and independently associated clinicopathologic characteristics. The predictive performances of the models were assessed with five-fold cross-validation. RESULTS Pretreatment SWE performed better than BUS in predicting the response to NAC for breast cancer for both the CNN and radiomics models (P < 0.001). The predictive results of the CNN models were significantly better than the radiomics models, with AUCs of 0.72 versus 0.69 for BUS and 0.80 versus 0.77 for SWE, respectively (P = 0.003). The CNN model based on the dual-modal US and molecular data exhibited outstanding performance in predicting NAC response, with an accuracy of 83.60% ± 2.63%, a sensitivity of 87.76% ± 6.44%, and a specificity of 77.45% ± 4.38%. CONCLUSION The pretreatment CNN model based on the dual-modal US and molecular data achieved excellent performance for predicting the response to chemotherapy in breast cancer. Therefore, this model has the potential to serve as a non-invasive objective biomarker to predict NAC response and aid clinicians with individual treatments.
Collapse
Affiliation(s)
- Jia-Xin Huang
- Department of Medical Ultrasound, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou 510000, China (J.-X.H., X.-Y.W., Y.-F.X., M.-J.W., L.-Z.L., X.-Q.P.)
| | - Jun Shi
- School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China (J.S., S.-S.D., H.-L.Z.)
| | - Sai-Sai Ding
- School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China (J.S., S.-S.D., H.-L.Z.)
| | - Hui-Li Zhang
- School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China (J.S., S.-S.D., H.-L.Z.)
| | - Xue-Yan Wang
- Department of Medical Ultrasound, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou 510000, China (J.-X.H., X.-Y.W., Y.-F.X., M.-J.W., L.-Z.L., X.-Q.P.)
| | - Shi-Yang Lin
- Department of Medical Ultrasound, The Sixth Affiliated Hospital of Sun Yat-sen University, Guangzhou 510000, China (S.-Y.L.)
| | - Yan-Fen Xu
- Department of Medical Ultrasound, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou 510000, China (J.-X.H., X.-Y.W., Y.-F.X., M.-J.W., L.-Z.L., X.-Q.P.)
| | - Ming-Jie Wei
- Department of Medical Ultrasound, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou 510000, China (J.-X.H., X.-Y.W., Y.-F.X., M.-J.W., L.-Z.L., X.-Q.P.)
| | - Long-Zhong Liu
- Department of Medical Ultrasound, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou 510000, China (J.-X.H., X.-Y.W., Y.-F.X., M.-J.W., L.-Z.L., X.-Q.P.)
| | - Xiao-Qing Pei
- Department of Medical Ultrasound, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou 510000, China (J.-X.H., X.-Y.W., Y.-F.X., M.-J.W., L.-Z.L., X.-Q.P.).
| |
Collapse
|
12
|
Zeng Q, Ke M, Zhong L, Zhou Y, Zhu X, He C, Liu L. Radiomics Based on Dynamic Contrast-Enhanced MRI to Early Predict Pathologic Complete Response in Breast Cancer Patients Treated with Neoadjuvant Therapy. Acad Radiol 2023; 30:1638-1647. [PMID: 36564256 DOI: 10.1016/j.acra.2022.11.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Revised: 10/31/2022] [Accepted: 11/10/2022] [Indexed: 12/24/2022]
Abstract
RATIONALE AND OBJECTIVES To investigate the value of dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI)-based radiomics at baseline and after two cycles of neoadjuvant therapy (NAT) and associated longitudinal changes for early prediction of the NAT response in patients with breast cancer. MATERIALS AND METHODS One hundred seventeen patients with breast cancer who underwent DCE-MRI before NAT and after two cycles of NAT from April 2019 to November 2021 were enrolled retrospectively. Patients were randomly divided into a training set (n = 81) and a test set (n = 36) at a ratio of 7:3. Clinical-pathological data and the relative tumor maximum diameter regression value (diameter%) were also collected. A total of 851 radiomic features were extracted from the phase with the most pronounced tumor enhancement on DCE-MRI T1 imaging acquired both pre- and post-treatment. Delta and delta% radiomics features were also calculated. The Least Absolute Shrinkage and Selection Operator (LASSO) method was applied to select features, and a logistic regression model was used to calculate pre-NAT, early-NAT, delta, and delta% radscores and then select among four radscores to build a Fusion radiomics model. The final clinical-radiomics model was constructed by combining fusion radscores and clinical-pathological variables. The discrimination and clinical utility of the models were further evaluated and compared. RESULTS The area under the curve (AUC) values of the fusion radiomics model based on pre-NAT, Delta, and Delta% radscores were 0.868 of 0.825. The clinical-radiomics model integrating Fusion radscores and clinical-pathological variables achieved AUC values of 0.920 of 0.884, which were higher than those of the clinical model constructed by AUC values (0.858/0.831), although no significant improvement was observed in the test set (Delong test, p = 0.196). Decision curve analysis (DCA) showed that the clinical-radiomics model demonstrated more clinical utility than the clinical model. CONCLUSION DCE-MRI-based radiomics features may have potential for pathological complete response (pCR) prediction in the early phase of NAT. By combining radiomics features and clinical-pathological characteristics, higher diagnostic performance can be achieved.
Collapse
Affiliation(s)
- Qiao Zeng
- Department of Radiology, Jiangxi Cancer Hospital, No. 519, Beijing East Road, Qingshanhu District Nanchang, Jiangxi Province, 330029, China (Q.Z., M.K., L.Z., Y.C., X.Z., L.L.); Department of Breast Surgery, Jiangxi Cancer Hospital, Nanchang, China (C.H.)
| | - Mengmeng Ke
- Department of Radiology, Jiangxi Cancer Hospital, No. 519, Beijing East Road, Qingshanhu District Nanchang, Jiangxi Province, 330029, China (Q.Z., M.K., L.Z., Y.C., X.Z., L.L.); Department of Breast Surgery, Jiangxi Cancer Hospital, Nanchang, China (C.H.)
| | - Linhua Zhong
- Department of Radiology, Jiangxi Cancer Hospital, No. 519, Beijing East Road, Qingshanhu District Nanchang, Jiangxi Province, 330029, China (Q.Z., M.K., L.Z., Y.C., X.Z., L.L.); Department of Breast Surgery, Jiangxi Cancer Hospital, Nanchang, China (C.H.)
| | - Yongjie Zhou
- Department of Radiology, Jiangxi Cancer Hospital, No. 519, Beijing East Road, Qingshanhu District Nanchang, Jiangxi Province, 330029, China (Q.Z., M.K., L.Z., Y.C., X.Z., L.L.); Department of Breast Surgery, Jiangxi Cancer Hospital, Nanchang, China (C.H.)
| | - Xuechao Zhu
- Department of Radiology, Jiangxi Cancer Hospital, No. 519, Beijing East Road, Qingshanhu District Nanchang, Jiangxi Province, 330029, China (Q.Z., M.K., L.Z., Y.C., X.Z., L.L.); Department of Breast Surgery, Jiangxi Cancer Hospital, Nanchang, China (C.H.)
| | - Chongwu He
- Department of Radiology, Jiangxi Cancer Hospital, No. 519, Beijing East Road, Qingshanhu District Nanchang, Jiangxi Province, 330029, China (Q.Z., M.K., L.Z., Y.C., X.Z., L.L.); Department of Breast Surgery, Jiangxi Cancer Hospital, Nanchang, China (C.H.)
| | - Lan Liu
- Department of Radiology, Jiangxi Cancer Hospital, No. 519, Beijing East Road, Qingshanhu District Nanchang, Jiangxi Province, 330029, China (Q.Z., M.K., L.Z., Y.C., X.Z., L.L.); Department of Breast Surgery, Jiangxi Cancer Hospital, Nanchang, China (C.H.).
| |
Collapse
|
13
|
Verma M, Abdelrahman L, Collado-Mesa F, Abdel-Mottaleb M. Multimodal Spatiotemporal Deep Learning Framework to Predict Response of Breast Cancer to Neoadjuvant Systemic Therapy. Diagnostics (Basel) 2023; 13:2251. [PMID: 37443648 DOI: 10.3390/diagnostics13132251] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Revised: 06/20/2023] [Accepted: 06/27/2023] [Indexed: 07/15/2023] Open
Abstract
Current approaches to breast cancer therapy include neoadjuvant systemic therapy (NST). The efficacy of NST is measured by pathologic complete response (pCR). A patient who attains pCR has significantly enhanced disease-free survival progress. The accurate prediction of pCR in response to a given treatment regimen could increase the likelihood of achieving pCR and prevent toxicities caused by treatments that are not effective. Th early prediction of response to NST can increase the likelihood of survival and help with decisions regarding breast-conserving surgery. An automated NST prediction framework that is able to precisely predict which patient undergoing NST will achieve a pathological complete response (pCR) at an early stage of treatment is needed. Here, we propose an end-to-end efficient multimodal spatiotemporal deep learning framework (deep-NST) framework to predict the outcome of NST prior or at an early stage of treatment. The deep-NST model incorporates imaging data captured at different timestamps of NST regimens, a tumor's molecular data, and a patient's demographic data. The efficacy of the proposed work is validated on the publicly available ISPY-1 dataset, in terms of accuracy, area under the curve (AUC), and computational complexity. In addition, seven ablation experiments were carried out to evaluate the impact of each design module in the proposed work. The experimental results show that the proposed framework performs significantly better than other recent methods.
Collapse
Affiliation(s)
- Monu Verma
- Department of Electrical and Computer Engineering, University of Miami, Miami, FL 33146, USA
| | | | - Fernando Collado-Mesa
- Department of Radiology, Miller School of Medicine, University of Miami, Miami, FL 33146, USA
| | - Mohamed Abdel-Mottaleb
- Department of Electrical and Computer Engineering, University of Miami, Miami, FL 33146, USA
| |
Collapse
|
14
|
Ozcan BB, Patel BK, Banerjee I, Dogan BE. Artificial Intelligence in Breast Imaging: Challenges of Integration Into Clinical Practice. JOURNAL OF BREAST IMAGING 2023; 5:248-257. [PMID: 38416888 DOI: 10.1093/jbi/wbad007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2022] [Indexed: 03/01/2024]
Abstract
Artificial intelligence (AI) in breast imaging is a rapidly developing field with promising results. Despite the large number of recent publications in this field, unanswered questions have led to limited implementation of AI into daily clinical practice for breast radiologists. This paper provides an overview of the key limitations of AI in breast imaging including, but not limited to, limited numbers of FDA-approved algorithms and annotated data sets with histologic ground truth; concerns surrounding data privacy, security, algorithm transparency, and bias; and ethical issues. Ultimately, the successful implementation of AI into clinical care will require thoughtful action to address these challenges, transparency, and sharing of AI implementation workflows, limitations, and performance metrics within the breast imaging community and other end-users.
Collapse
Affiliation(s)
- B Bersu Ozcan
- The University of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX, USA
| | | | - Imon Banerjee
- Mayo Clinic, Department of Radiology, Scottsdale, AZ, USA
| | - Basak E Dogan
- The University of Texas Southwestern Medical Center, Department of Radiology, Dallas, TX, USA
| |
Collapse
|
15
|
Wu G, Cheligeer C, Brisson AM, Quan ML, Cheung WY, Brenner D, Lupichuk S, Teman C, Basmadjian RB, Popwich B, Xu Y. A New Method of Identifying Pathologic Complete Response After Neoadjuvant Chemotherapy for Breast Cancer Patients Using a Population-Based Electronic Medical Record System. Ann Surg Oncol 2023; 30:2095-2103. [PMID: 36542249 DOI: 10.1245/s10434-022-12955-6] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2022] [Accepted: 12/01/2022] [Indexed: 12/24/2022]
Abstract
BACKGROUND Accurate identification of pathologic complete response (pCR) from population-based electronic narrative data in a timely and cost-efficient manner is critical. This study aimed to derive and validate a set of natural language processing (NLP)-based machine-learning algorithms to capture pCR from surgical pathology reports of breast cancer patients who underwent neoadjuvant chemotherapy (NAC). METHODS This retrospective cohort study included all invasive breast cancer patients who underwent NAC and subsequent curative-intent surgery during their admission at all four tertiary acute care hospitals in Calgary, Alberta, Canada, between 1 January 2010 and 31 December 2017. Surgical pathology reports were extracted and processed with NLP. Decision tree classifiers were constructed and validated against chart review results. Machine-learning algorithms were evaluated with a performance matrix including sensitivity, specificity, positive predictive value (PPV), negative predictive value [NPV], accuracy, area under the receiver operating characteristic curve [AUC], and F1 score. RESULTS The study included 351 female patients. Of these patients, 102 (29%) achieved pCR after NAC. The high-sensitivity model achieved a sensitivity of 90.5% (95% confidence interval [CI], 69.6-98.9%), a PPV of 76% (95% CI, 59.6-87.2), an accuracy of 88.6% (95% CI, 78.7-94.9%), an AUC of 0.891 (95% CI, 0.795-0.987), and an F1 score of 82.61. The high-PPV algorithm reached a sensitivity of 85.7% (95% CI, 63.7-97%), a PPV of 81.8% (95% CI, 63.4-92.1%), an accuracy of 90% (95% CI, 80.5-95.9%), an AUC of 0.888 (95% CI, 0.790-0.985), and an F1 score of 83.72. The high-F1 score algorithm obtained a performance equivalent to that of the high-PPV algorithm. CONCLUSION The developed algorithms demonstrated excellent accuracy in identifying pCR from surgical pathology reports of breast cancer patients who received NAC treatment.
Collapse
Affiliation(s)
- Guosong Wu
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
- The Centre for Health Informatics, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Cheligeer Cheligeer
- The Centre for Health Informatics, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
- Alberta Health Services, Calgary, AB, Canada
| | - Anne-Marie Brisson
- Departments of Oncology, Community Health Sciences, and Surgery, and The Center for Health Informatics, Cumming School of Medicine, University of Calgary, 3280 Hospital Drive NW, Calgary, AB, T2N4Z6, Canada
| | - May Lynn Quan
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
- Departments of Oncology, Community Health Sciences, and Surgery, and The Center for Health Informatics, Cumming School of Medicine, University of Calgary, 3280 Hospital Drive NW, Calgary, AB, T2N4Z6, Canada
- Department of Surgery, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Winson Y Cheung
- Department of Radiology, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
- Department of Surgery, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Darren Brenner
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
- Departments of Oncology, Community Health Sciences, and Surgery, and The Center for Health Informatics, Cumming School of Medicine, University of Calgary, 3280 Hospital Drive NW, Calgary, AB, T2N4Z6, Canada
| | - Sasha Lupichuk
- Departments of Oncology, Community Health Sciences, and Surgery, and The Center for Health Informatics, Cumming School of Medicine, University of Calgary, 3280 Hospital Drive NW, Calgary, AB, T2N4Z6, Canada
| | - Carolin Teman
- Department of Pathology and Laboratory Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Robert Barkev Basmadjian
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Brittany Popwich
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
- Departments of Oncology, Community Health Sciences, and Surgery, and The Center for Health Informatics, Cumming School of Medicine, University of Calgary, 3280 Hospital Drive NW, Calgary, AB, T2N4Z6, Canada
- Department of Surgery, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Yuan Xu
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.
- Departments of Oncology, Community Health Sciences, and Surgery, and The Center for Health Informatics, Cumming School of Medicine, University of Calgary, 3280 Hospital Drive NW, Calgary, AB, T2N4Z6, Canada.
- Department of Surgery, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.
| |
Collapse
|
16
|
Zhao X, Bai JW, Guo Q, Ren K, Zhang GJ. Clinical applications of deep learning in breast MRI. Biochim Biophys Acta Rev Cancer 2023; 1878:188864. [PMID: 36822377 DOI: 10.1016/j.bbcan.2023.188864] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2022] [Revised: 01/05/2023] [Accepted: 01/17/2023] [Indexed: 02/25/2023]
Abstract
Deep learning (DL) is one of the most powerful data-driven machine-learning techniques in artificial intelligence (AI). It can automatically learn from raw data without manual feature selection. DL models have led to remarkable advances in data extraction and analysis for medical imaging. Magnetic resonance imaging (MRI) has proven useful in delineating the characteristics and extent of breast lesions and tumors. This review summarizes the current state-of-the-art applications of DL models in breast MRI. Many recent DL models were examined in this field, along with several advanced learning approaches and methods for data normalization and breast and lesion segmentation. For clinical applications, DL-based breast MRI models were proven useful in five aspects: diagnosis of breast cancer, classification of molecular types, classification of histopathological types, prediction of neoadjuvant chemotherapy response, and prediction of lymph node metastasis. For subsequent studies, further improvement in data acquisition and preprocessing is necessary, additional DL techniques in breast MRI should be investigated, and wider clinical applications need to be explored.
Collapse
Affiliation(s)
- Xue Zhao
- Fujian Key Laboratory of Precision Diagnosis and Treatment in Breast Cancer, Xiang'an Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China; National Institute for Data Science in Health and Medicine, Xiamen University, Xiamen, China; Department of Breast-Thyroid-Surgery and Cancer Center, Xiang'an Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China; Xiamen Research Center of Clinical Medicine in Breast & Thyroid Cancers, Xiang'an Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China; Xiamen Key Laboratory of Endocrine-Related Cancer Precision Medicine, Xiang'an Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China
| | - Jing-Wen Bai
- Fujian Key Laboratory of Precision Diagnosis and Treatment in Breast Cancer, Xiang'an Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China; Xiamen Research Center of Clinical Medicine in Breast & Thyroid Cancers, Xiang'an Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China; Xiamen Key Laboratory of Endocrine-Related Cancer Precision Medicine, Xiang'an Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China; Department of Oncology, Xiang'an Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China; Cancer Research Center, School of Medicine, Xiamen University, Xiamen, China
| | - Qiu Guo
- Department of Radiology, Xiang'an Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China
| | - Ke Ren
- Department of Radiology, Xiang'an Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China.
| | - Guo-Jun Zhang
- Fujian Key Laboratory of Precision Diagnosis and Treatment in Breast Cancer, Xiang'an Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China; Department of Breast-Thyroid-Surgery and Cancer Center, Xiang'an Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China; Xiamen Research Center of Clinical Medicine in Breast & Thyroid Cancers, Xiang'an Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China; Xiamen Key Laboratory of Endocrine-Related Cancer Precision Medicine, Xiang'an Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China; Cancer Research Center, School of Medicine, Xiamen University, Xiamen, China.
| |
Collapse
|
17
|
Huang Z, Shao W, Han Z, Alkashash AM, De la Sancha C, Parwani AV, Nitta H, Hou Y, Wang T, Salama P, Rizkalla M, Zhang J, Huang K, Li Z. Artificial intelligence reveals features associated with breast cancer neoadjuvant chemotherapy responses from multi-stain histopathologic images. NPJ Precis Oncol 2023; 7:14. [PMID: 36707660 PMCID: PMC9883475 DOI: 10.1038/s41698-023-00352-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2021] [Accepted: 01/16/2023] [Indexed: 01/28/2023] Open
Abstract
Advances in computational algorithms and tools have made the prediction of cancer patient outcomes using computational pathology feasible. However, predicting clinical outcomes from pre-treatment histopathologic images remains a challenging task, limited by the poor understanding of tumor immune micro-environments. In this study, an automatic, accurate, comprehensive, interpretable, and reproducible whole slide image (WSI) feature extraction pipeline known as, IMage-based Pathological REgistration and Segmentation Statistics (IMPRESS), is described. We used both H&E and multiplex IHC (PD-L1, CD8+, and CD163+) images, investigated whether artificial intelligence (AI)-based algorithms using automatic feature extraction methods can predict neoadjuvant chemotherapy (NAC) outcomes in HER2-positive (HER2+) and triple-negative breast cancer (TNBC) patients. Features are derived from tumor immune micro-environment and clinical data and used to train machine learning models to accurately predict the response to NAC in breast cancer patients (HER2+ AUC = 0.8975; TNBC AUC = 0.7674). The results demonstrate that this method outperforms the results trained from features that were manually generated by pathologists. The developed image features and algorithms were further externally validated by independent cohorts, yielding encouraging results, especially for the HER2+ subtype.
Collapse
Affiliation(s)
- Zhi Huang
- School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, 47907, USA
- Department of Electrical and Computer Engineering, Indiana University - Purdue University Indianapolis, Indianapolis, IN, 46202, USA
| | - Wei Shao
- Department of Medicine, Indiana University School of Medicine, Indianapolis, IN, 46202, USA
| | - Zhi Han
- Department of Medicine, Indiana University School of Medicine, Indianapolis, IN, 46202, USA
- Regenstrief Institute, Indianapolis, IN, 46202, USA
- Department of Biostatistics and Health Data Science, Indiana University School of Medicine, Indianapolis, IN, 46202, USA
| | - Ahmad Mahmoud Alkashash
- Department of Pathology, Indiana University School of Medicine, Indianapolis, IN, 46202, USA
| | - Carlo De la Sancha
- Department of Pathology, Indiana University School of Medicine, Indianapolis, IN, 46202, USA
| | - Anil V Parwani
- Department of Pathology, The Ohio State University Wexner Medical Center, Columbus, OH, 43210, USA
| | - Hiroaki Nitta
- Roche Tissue Diagnostics, 1910 E. Innovation Park Drive, Tucson, AZ, 85755, USA
| | - Yanjun Hou
- University Hospitals Cleveland Medical Center, Case Western Reserve University, 11100 Euclid Avenue, Cleveland, OH, 44106, USA
| | - Tongxin Wang
- Department of Computer Science, Indiana University Bloomington, Bloomington, IN, 47408, USA
| | - Paul Salama
- Department of Electrical and Computer Engineering, Indiana University - Purdue University Indianapolis, Indianapolis, IN, 46202, USA
| | - Maher Rizkalla
- Department of Electrical and Computer Engineering, Indiana University - Purdue University Indianapolis, Indianapolis, IN, 46202, USA
| | - Jie Zhang
- Department of Medical and Molecular Genetics, Indiana University School of Medicine, Indianapolis, IN, 46202, USA
| | - Kun Huang
- Department of Medicine, Indiana University School of Medicine, Indianapolis, IN, 46202, USA.
- Regenstrief Institute, Indianapolis, IN, 46202, USA.
- Department of Biostatistics and Health Data Science, Indiana University School of Medicine, Indianapolis, IN, 46202, USA.
| | - Zaibo Li
- Department of Pathology, The Ohio State University Wexner Medical Center, Columbus, OH, 43210, USA.
| |
Collapse
|
18
|
Prediction of pathologic complete response to neoadjuvant systemic therapy in triple negative breast cancer using deep learning on multiparametric MRI. Sci Rep 2023; 13:1171. [PMID: 36670144 PMCID: PMC9859781 DOI: 10.1038/s41598-023-27518-2] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2022] [Accepted: 01/03/2023] [Indexed: 01/22/2023] Open
Abstract
Triple-negative breast cancer (TNBC) is an aggressive subtype of breast cancer. Neoadjuvant systemic therapy (NAST) followed by surgery are currently standard of care for TNBC with 50-60% of patients achieving pathologic complete response (pCR). We investigated ability of deep learning (DL) on dynamic contrast enhanced (DCE) MRI and diffusion weighted imaging acquired early during NAST to predict TNBC patients' pCR status in the breast. During the development phase using the images of 130 TNBC patients, the DL model achieved areas under the receiver operating characteristic curves (AUCs) of 0.97 ± 0.04 and 0.82 ± 0.10 for the training and the validation, respectively. The model achieved an AUC of 0.86 ± 0.03 when evaluated in the independent testing group of 32 patients. In an additional prospective blinded testing group of 48 patients, the model achieved an AUC of 0.83 ± 0.02. These results demonstrated that DL based on multiparametric MRI can potentially differentiate TNBC patients with pCR or non-pCR in the breast early during NAST.
Collapse
|
19
|
Machine learning with textural analysis of longitudinal multiparametric MRI and molecular subtypes accurately predicts pathologic complete response in patients with invasive breast cancer. PLoS One 2023; 18:e0280320. [PMID: 36649274 PMCID: PMC9844845 DOI: 10.1371/journal.pone.0280320] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2022] [Accepted: 12/27/2022] [Indexed: 01/18/2023] Open
Abstract
PURPOSE To predict pathological complete response (pCR) after neoadjuvant chemotherapy using extreme gradient boosting (XGBoost) with MRI and non-imaging data at multiple treatment timepoints. MATERIAL AND METHODS This retrospective study included breast cancer patients (n = 117) who underwent neoadjuvant chemotherapy. Data types used included tumor ADC values, diffusion-weighted and dynamic-contrast-enhanced MRI at three treatment timepoints, and patient demographics and tumor data. GLCM textural analysis was performed on MRI data. An extreme gradient boosting machine learning algorithm was used to predict pCR. Prediction performance was evaluated using the area under the curve (AUC) of the receiver operating curve along with precision and recall. RESULTS Prediction using texture features of DWI and DCE images at multiple treatment time points (AUC = 0.871; 95% CI: (0.768, 0.974; p<0.001) and (AUC = 0.903 95% CI: 0.854, 0.952; p<0.001) respectively), outperformed that using mean tumor ADC (AUC = 0.850 (95% CI: 0.764, 0.936; p<0.001)). The AUC using all MRI data was 0.933 (95% CI: 0.836, 1.03; p<0.001). The AUC using non-MRI data was 0.919 (95% CI: 0.848, 0.99; p<0.001). The highest AUC of 0.951 (95% CI: 0.909, 0.993; p<0.001) was achieved with all MRI and all non-MRI data at all time points as inputs. CONCLUSION Using XGBoost on extracted GLCM features and non-imaging data accurately predicts pCR. This early prediction of response can minimize exposure to toxic chemotherapy, allowing regimen modification mid-treatment and ultimately achieving better outcomes.
Collapse
|
20
|
Dammu H, Ren T, Duong TQ. Deep learning prediction of pathological complete response, residual cancer burden, and progression-free survival in breast cancer patients. PLoS One 2023; 18:e0280148. [PMID: 36607982 PMCID: PMC9821469 DOI: 10.1371/journal.pone.0280148] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Accepted: 12/20/2022] [Indexed: 01/07/2023] Open
Abstract
The goal of this study was to employ novel deep-learning convolutional-neural-network (CNN) to predict pathological complete response (PCR), residual cancer burden (RCB), and progression-free survival (PFS) in breast cancer patients treated with neoadjuvant chemotherapy using longitudinal multiparametric MRI, demographics, and molecular subtypes as inputs. In the I-SPY-1 TRIAL, 155 patients with stage 2 or 3 breast cancer with breast tumors underwent neoadjuvant chemotherapy met the inclusion/exclusion criteria. The inputs were dynamic-contrast-enhanced (DCE) MRI, and T2- weighted MRI as three-dimensional whole-images without the tumor segmentation, as well as molecular subtypes and demographics. The outcomes were PCR, RCB, and PFS. Three ("Integrated", "Stack" and "Concatenation") CNN were evaluated using receiver-operating characteristics and mean absolute errors. The Integrated approach outperformed the "Stack" or "Concatenation" CNN. Inclusion of both MRI and non-MRI data outperformed either alone. The combined pre- and post-neoadjuvant chemotherapy data outperformed either alone. Using the best model and data combination, PCR prediction yielded an accuracy of 0.81±0.03 and AUC of 0.83±0.03; RCB prediction yielded an accuracy of 0.80±0.02 and Cohen's κ of 0.73±0.03; PFS prediction yielded a mean absolute error of 24.6±0.7 months (survival ranged from 6.6 to 127.5 months). Deep learning using longitudinal multiparametric MRI, demographics, and molecular subtypes accurately predicts PCR, RCB, and PFS in breast cancer patients. This approach may prove useful for treatment selection, planning, execution, and mid-treatment adjustment.
Collapse
Affiliation(s)
- Hongyi Dammu
- Department of Radiology, Montefiore Medical Center and Albert Einstein College of Medicine, Bronx, New York, United States of America
| | - Thomas Ren
- Department of Radiology, Montefiore Medical Center and Albert Einstein College of Medicine, Bronx, New York, United States of America
| | - Tim Q. Duong
- Department of Radiology, Montefiore Medical Center and Albert Einstein College of Medicine, Bronx, New York, United States of America
- * E-mail:
| |
Collapse
|
21
|
Zhang X, Liu M, Ren W, Sun J, Wang K, Xi X, Zhang G. Predicting of axillary lymph node metastasis in invasive breast cancer using multiparametric MRI dataset based on CNN model. Front Oncol 2022; 12:1069733. [PMID: 36561533 PMCID: PMC9763602 DOI: 10.3389/fonc.2022.1069733] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Accepted: 11/15/2022] [Indexed: 12/12/2022] Open
Abstract
Purpose To develop a multiparametric MRI model for predicting axillary lymph node metastasis in invasive breast cancer. Methods Clinical data and T2WI, DWI, and DCE-MRI images of 252 patients with invasive breast cancer were retrospectively analyzed and divided into the axillary lymph node metastasis (ALNM) group and non-ALNM group using biopsy results as a reference standard. The regions of interest (ROI) in T2WI, DWI, and DCE-MRI images were segmented using MATLAB software, and the ROI was unified into 224 × 224 sizes, followed by image normalization as input to T2WI, DWI, and DCE-MRI models, all of which were based on ResNet 50 networks. The idea of a weighted voting method in ensemble learning was employed, and then T2WI, DWI, and DCE-MRI models were used as the base models to construct a multiparametric MRI model. The entire dataset was randomly divided into training sets and testing sets (the training set 202 cases, including 78 ALNM, 124 non-ALNM; the testing set 50 cases, including 20 ALNM, 30 non-ALNM). Accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of models were calculated. The receiver operating characteristic (ROC) curve and area under the curve (AUC) were used to evaluate the diagnostic performance of each model for axillary lymph node metastasis, and the DeLong test was performed, P< 0.05 statistically significant. Results For the assessment of axillary lymph node status in invasive breast cancer on the test set, multiparametric MRI models yielded an AUC of 0.913 (95% CI, 0.799-0.974); T2WI-based model yielded an AUC of 0.908 (95% CI, 0.792-0.971); DWI-based model achieved an AUC of 0.702 (95% CI, 0.556-0.823); and the AUC of the DCE-MRI-based model was 0.572 (95% CI, 0.424-0.711). The improvement in the diagnostic performance of the multiparametric MRI model compared with the DWI and DCE-MRI-based models were significant (P< 0.01 for both). However, the increase was not meaningful compared with the T2WI-based model (P = 0.917). Conclusion Multiparametric MRI image analysis based on an ensemble CNN model with deep learning is of practical application and extension for preoperative prediction of axillary lymph node metastasis in invasive breast cancer.
Collapse
Affiliation(s)
- Xiaodong Zhang
- Department of Radiology, The First Affiliated Hospital of Shandong First Medical University, Jinan, China,Postgraduate Department, Shandong First Medical University (Shandong Academy of Medical Sciences), Jinan, China
| | - Menghan Liu
- Department of Health Management, The First Affiliated Hospital of Shandong First Medical University, Jinan, China
| | - Wanqing Ren
- Department of Radiology, The First Affiliated Hospital of Shandong First Medical University, Jinan, China,Postgraduate Department, Shandong First Medical University (Shandong Academy of Medical Sciences), Jinan, China
| | - Jingxiang Sun
- Department of Radiology, The First Affiliated Hospital of Shandong First Medical University, Jinan, China,Postgraduate Department, Shandong First Medical University (Shandong Academy of Medical Sciences), Jinan, China
| | - Kesong Wang
- School of Computer Science and Technology, Shandong Jianzhu University, Jinan, China
| | - Xiaoming Xi
- School of Computer Science and Technology, Shandong Jianzhu University, Jinan, China
| | - Guang Zhang
- Department of Health Management, The First Affiliated Hospital of Shandong First Medical University, Jinan, China,*Correspondence: Guang Zhang,
| |
Collapse
|
22
|
O'Donnell J, Gasior S, Davey M, O'Malley E, Lowery A, McGarry J, O'Connell A, Kerin M, McCarthy P. The accuracy of breast MRI radiomic methodologies in predicting pathological complete response to neoadjuvant chemotherapy: A systematic review and network meta-analysis. Eur J Radiol 2022; 157:110561. [DOI: 10.1016/j.ejrad.2022.110561] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2022] [Revised: 09/13/2022] [Accepted: 10/11/2022] [Indexed: 11/03/2022]
|
23
|
Khan N, Adam R, Huang P, Maldjian T, Duong TQ. Deep Learning Prediction of Pathologic Complete Response in Breast Cancer Using MRI and Other Clinical Data: A Systematic Review. Tomography 2022; 8:2784-2795. [PMID: 36412691 PMCID: PMC9680498 DOI: 10.3390/tomography8060232] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2022] [Revised: 11/12/2022] [Accepted: 11/16/2022] [Indexed: 11/24/2022] Open
Abstract
Breast cancer patients who have pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) are more likely to have better clinical outcomes. The ability to predict which patient will respond to NAC early in the treatment course is important because it could help to minimize unnecessary toxic NAC and to modify regimens mid-treatment to achieve better efficacy. Machine learning (ML) is increasingly being used in radiology and medicine because it can identify relationships amongst complex data elements to inform outcomes without the need to specify such relationships a priori. One of the most popular deep learning methods that applies to medical images is the Convolutional Neural Networks (CNN). In contrast to supervised ML, deep learning CNN can operate on the whole images without requiring radiologists to manually contour the tumor on images. Although there have been many review papers on supervised ML prediction of pCR, review papers on deep learning prediction of pCR are sparse. Deep learning CNN could also incorporate multiple image types, clinical data such as demographics and molecular subtypes, as well as data from multiple treatment time points to predict pCR. The goal of this study is to perform a systematic review of deep learning methods that use whole-breast MRI images without annotation or tumor segmentation to predict pCR in breast cancer.
Collapse
|
24
|
Tian G, Xu D, He Y, Chai W, Deng Z, Cheng C, Jin X, Wei G, Zhao Q, Jiang T. Deep learning for real-time auxiliary diagnosis of pancreatic cancer in endoscopic ultrasonography. Front Oncol 2022; 12:973652. [PMID: 36276094 PMCID: PMC9586286 DOI: 10.3389/fonc.2022.973652] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Accepted: 09/21/2022] [Indexed: 11/13/2022] Open
Abstract
In recent year, many deep learning have been playing an important role in the detection of cancers. This study aimed to real-timely differentiate a pancreatic cancer (PC) or a non-pancreatic cancer (NPC) lesion via endoscopic ultrasonography (EUS) image. A total of 1213 EUS images from 157 patients (99 male, 58 female) with pancreatic disease were used for training, validation and test groups. Before model training, regions of interest (ROIs) were manually drawn to mark the PC and NPC lesions using Labelimage software. Yolov5m was used as the algorithm model to automatically distinguish the presence of pancreatic lesion. After training the model based on EUS images using YOLOv5, the parameters achieved convergence within 300 rounds (GIoU Loss: 0.01532, Objectness Loss: 0.01247, precision: 0.713 and recall: 0.825). For the validation group, the mAP0.5 was 0.831, and mAP@.5:.95 was 0.512. In addition, the receiver operating characteristic (ROC) curve analysis showed this model seemed to have a trend of more AUC of 0.85 (0.665 to 0.956) than the area under the curve (AUC) of 0.838 (0.65 to 0.949) generated by physicians using EUS detection without puncture, although pairwise comparison of ROC curves showed that the AUC between the two groups was not significant (z= 0.15, p = 0.8804). This study suggested that the YOLOv5m would generate attractive results and allow for the real-time decision support for distinction of a PC or a NPC lesion.
Collapse
Affiliation(s)
- Guo Tian
- Department of Ultrasound Medicine, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
- Key Laboratory of Pulsed Power Translational Medicine of Zhejiang Province, Hangzhou, China
- State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Danxia Xu
- Department of Ultrasound Medicine, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
- Key Laboratory of Pulsed Power Translational Medicine of Zhejiang Province, Hangzhou, China
| | - Yinghua He
- Department of Clinical Pharmacy, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
- Zhejiang Provincial Key Laboratory for Drug Evaluation and Clinical Research, Hangzhou, China
| | - Weilu Chai
- Department of Ultrasound Medicine, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
- Key Laboratory of Pulsed Power Translational Medicine of Zhejiang Province, Hangzhou, China
| | - Zhuang Deng
- Department of Ultrasound Medicine, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Chao Cheng
- Department of Ultrasound Medicine, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Xinyan Jin
- Department of Ultrasound Medicine, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Guyue Wei
- Department of Ultrasound Medicine, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Qiyu Zhao
- Department of Ultrasound Medicine, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
- Key Laboratory of Pulsed Power Translational Medicine of Zhejiang Province, Hangzhou, China
| | - Tianan Jiang
- Department of Ultrasound Medicine, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
- Key Laboratory of Pulsed Power Translational Medicine of Zhejiang Province, Hangzhou, China
- Zhejiang University Cancer Center, Hangzhou, China
- *Correspondence: Tianan Jiang,
| |
Collapse
|
25
|
Kim SY, Cho N. Breast Magnetic Resonance Imaging for Patients With Newly Diagnosed Breast Cancer: A Review. J Breast Cancer 2022; 25:263-277. [PMID: 36031752 PMCID: PMC9411024 DOI: 10.4048/jbc.2022.25.e35] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Revised: 07/13/2022] [Accepted: 07/25/2022] [Indexed: 11/30/2022] Open
Abstract
Despite the high sensitivity and widespread use of preoperative magnetic resonance imaging (MRI), the American Cancer Society and the National Comprehensive Cancer Network guidelines do not recommend the routine use of preoperative MRI owing to the conflicting results and lack of clear benefit to the surgical outcome (reoperation and mastectomy) and long-term clinical outcomes (local recurrence and metachronous contralateral breast cancer). Preoperative MRI detects additional cancers that are occult at mammography and ultrasound but increases the rate of mastectomy. Concerns about overdiagnosis and overtreatment of preoperative MRI might be mitigated by adjusting the confounding factors when conducting studies, using the state-of-the-art image-guided biopsy technique, applying the radiologists’ cumulative experiences in interpreting MRI findings, and performing multiple lumpectomies in patients with multicentric cancer. Among the various imaging methods, dynamic contrast-enhanced MRI has the highest accuracy in predicting pathologic complete response after neoadjuvant chemotherapy. Prospective trials aimed at applying the MRI information to the de-escalation of surgical or radiation treatments are underway. In this review, current studies on the clinical outcomes of preoperative breast MRI are updated, and circumstances in which MRI may be useful for surgical planning are discussed.
Collapse
Affiliation(s)
- Soo-Yeon Kim
- Department of Radiology, Seoul National University Hospital, Seoul, Korea.,Department of Radiology, Seoul National University College of Medicine, Seoul, Korea.,Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea
| | - Nariya Cho
- Department of Radiology, Seoul National University Hospital, Seoul, Korea.,Department of Radiology, Seoul National University College of Medicine, Seoul, Korea.,Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea.
| |
Collapse
|
26
|
A Survey on Deep Learning for Precision Oncology. Diagnostics (Basel) 2022; 12:diagnostics12061489. [PMID: 35741298 PMCID: PMC9222056 DOI: 10.3390/diagnostics12061489] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Revised: 06/14/2022] [Accepted: 06/14/2022] [Indexed: 12/27/2022] Open
Abstract
Precision oncology, which ensures optimized cancer treatment tailored to the unique biology of a patient’s disease, has rapidly developed and is of great clinical importance. Deep learning has become the main method for precision oncology. This paper summarizes the recent deep-learning approaches relevant to precision oncology and reviews over 150 articles within the last six years. First, we survey the deep-learning approaches categorized by various precision oncology tasks, including the estimation of dose distribution for treatment planning, survival analysis and risk estimation after treatment, prediction of treatment response, and patient selection for treatment planning. Secondly, we provide an overview of the studies per anatomical area, including the brain, bladder, breast, bone, cervix, esophagus, gastric, head and neck, kidneys, liver, lung, pancreas, pelvis, prostate, and rectum. Finally, we highlight the challenges and discuss potential solutions for future research directions.
Collapse
|
27
|
Liang X, Yu X, Gao T. Machine learning with magnetic resonance imaging for prediction of response to neoadjuvant chemotherapy in breast cancer: A systematic review and meta-analysis. Eur J Radiol 2022; 150:110247. [DOI: 10.1016/j.ejrad.2022.110247] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2021] [Revised: 02/14/2022] [Accepted: 03/07/2022] [Indexed: 12/18/2022]
|
28
|
Xie J, Shi H, Du C, Song X, Wei J, Dong Q, Wan C. Dual-Branch Convolutional Neural Network Based on Ultrasound Imaging in the Early Prediction of Neoadjuvant Chemotherapy Response in Patients With Locally Advanced Breast Cancer. Front Oncol 2022; 12:812463. [PMID: 35463368 PMCID: PMC9026439 DOI: 10.3389/fonc.2022.812463] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2021] [Accepted: 03/07/2022] [Indexed: 11/13/2022] Open
Abstract
The early prediction of a patient’s response to neoadjuvant chemotherapy (NAC) in breast cancer treatment is crucial for guiding therapy decisions. We aimed to develop a novel approach, named the dual-branch convolutional neural network (DBNN), based on deep learning that uses ultrasound (US) images for the early prediction of NAC response in patients with locally advanced breast cancer (LABC). This retrospective study included 114 women who were monitored with US during pretreatment (NAC pre) and after one cycle of NAC (NAC1). Pathologic complete response (pCR) was defined as no residual invasive carcinoma in the breast. For predicting pCR, the data were randomly split into a training set and test set (4:1). DBNN with US images was proposed to predict pCR early in breast cancer patients who received NAC. The connection between pretreatment data and data obtained after the first cycle of NAC was considered through the feature sharing of different branches. Moreover, the importance of data in various stages was emphasized by changing the weight of the two paths to classify those with pCR. The optimal model architecture of DBNN was determined by two ablation experiments. The diagnostic performance of DBNN for predicting pCR was compared with that of four methods from the latest research. To further validate the potential of DBNN in the early prediction of NAC response, the data from NAC pre and NAC1 were separately assessed. In the prediction of pCR, the highest diagnostic performance was obtained when combining the US image information of NAC pre and NAC1 (area under the receiver operating characteristic curve (AUC): 0.939; 95% confidence interval (CI): 0.907, 0.972; F1-score: 0.850; overall accuracy: 87.5%; sensitivity: 90.67%; and specificity: 85.67%), and the diagnostic performance with the combined data was superior to the performance when only NAC pre (AUC: 0.730; 95% CI: 0.657, 0.802; F1-score: 0.675; sensitivity: 76.00%; and specificity: 68.38%) or NAC1 (AUC: 0.739; 95% CI: 0.664, 0.813; F1-score: 0.611; sensitivity: 53.33%; and specificity: 86.32%) (p<0.01) was used. As a noninvasive prediction tool, DBNN can achieve outstanding results in the early prediction of NAC response in patients with LABC when combining the US data of NAC pre and NAC1.
Collapse
Affiliation(s)
- Jiang Xie
- School of Computer Engineering and Science, Shanghai University, Shanghai, China
| | - Huachan Shi
- School of Computer Engineering and Science, Shanghai University, Shanghai, China
| | - Chengrun Du
- Department of Radiation Oncology, Fudan University Shanghai Cancer Center, Shanghai, China
| | - Xiangshuai Song
- School of Computer Engineering and Science, Shanghai University, Shanghai, China
| | - Jinzhu Wei
- School of Medicine, Shanghai University, Shanghai, China
| | - Qi Dong
- Department of Ultrasound, Ren Ji Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Caifeng Wan
- Department of Ultrasound, Ren Ji Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| |
Collapse
|
29
|
Balkenende L, Teuwen J, Mann RM. Application of Deep Learning in Breast Cancer Imaging. Semin Nucl Med 2022; 52:584-596. [PMID: 35339259 DOI: 10.1053/j.semnuclmed.2022.02.003] [Citation(s) in RCA: 32] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2022] [Revised: 02/15/2022] [Accepted: 02/16/2022] [Indexed: 11/11/2022]
Abstract
This review gives an overview of the current state of deep learning research in breast cancer imaging. Breast imaging plays a major role in detecting breast cancer at an earlier stage, as well as monitoring and evaluating breast cancer during treatment. The most commonly used modalities for breast imaging are digital mammography, digital breast tomosynthesis, ultrasound and magnetic resonance imaging. Nuclear medicine imaging techniques are used for detection and classification of axillary lymph nodes and distant staging in breast cancer imaging. All of these techniques are currently digitized, enabling the possibility to implement deep learning (DL), a subset of Artificial intelligence, in breast imaging. DL is nowadays embedded in a plethora of different tasks, such as lesion classification and segmentation, image reconstruction and generation, cancer risk prediction, and prediction and assessment of therapy response. Studies show similar and even better performances of DL algorithms compared to radiologists, although it is clear that large trials are needed, especially for ultrasound and magnetic resonance imaging, to exactly determine the added value of DL in breast cancer imaging. Studies on DL in nuclear medicine techniques are only sparsely available and further research is mandatory. Legal and ethical issues need to be considered before the role of DL can expand to its full potential in clinical breast care practice.
Collapse
Affiliation(s)
- Luuk Balkenende
- Department of Radiology, Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands; Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jonas Teuwen
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands; Department of Radiation Oncology, Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | - Ritse M Mann
- Department of Radiology, Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands; Department of Medical Imaging, Radboud University Medical Center, Nijmegen, The Netherlands.
| |
Collapse
|
30
|
Peng Y, Cheng Z, Gong C, Zheng C, Zhang X, Wu Z, Yang Y, Yang X, Zheng J, Shen J. Pretreatment DCE-MRI-Based Deep Learning Outperforms Radiomics Analysis in Predicting Pathologic Complete Response to Neoadjuvant Chemotherapy in Breast Cancer. Front Oncol 2022; 12:846775. [PMID: 35359387 PMCID: PMC8960929 DOI: 10.3389/fonc.2022.846775] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2021] [Accepted: 01/26/2022] [Indexed: 01/02/2023] Open
Abstract
PurposeTo compare the performances of deep learning (DL) to radiomics analysis (RA) in predicting pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) based on pretreatment dynamic contrast-enhanced MRI (DCE-MRI) in breast cancer.Materials and MethodsThis retrospective study included 356 breast cancer patients who underwent DCE-MRI before NAC and underwent surgery after NAC. Image features and kinetic parameters of tumors were derived from DCE-MRI. Molecular information was assessed based on immunohistochemistry results. The image-based RA and DL models were constructed by adding kinetic parameters or molecular information to image-only linear discriminant analysis (LDA) and convolutional neural network (CNN) models. The predictive performances of developed models were assessed by receiver operating characteristic (ROC) curve analysis and compared with the DeLong method.ResultsThe overall pCR rate was 23.3% (83/356). The area under the ROC (AUROC) of the image-kinetic-molecular RA model was 0.781 [95% confidence interval (CI): 0.735, 0.828], which was higher than that of the image-kinetic RA model (0.629, 95% CI: 0.595, 0.663; P < 0.001) and comparable to that of the image-molecular RA model (0.755, 95% CI: 0.708, 0.802; P = 0.133). The AUROC of the image-kinetic-molecular DL model was 0.83 (95% CI: 0.816, 0.847), which was higher than that of the image-kinetic and image-molecular DL models (0.707, 95% CI: 0.654, 0.761; 0.79, 95% CI: 0.768, 0.812; P < 0.001) and higher than that of the image-kinetic-molecular RA model (0.778, 95% CI: 0.735, 0.828; P < 0.001).ConclusionsThe pretreatment DCE-MRI-based DL model is superior to the RA model in predicting pCR to NAC in breast cancer patients. The image-kinetic-molecular DL model has the best prediction performance.
Collapse
Affiliation(s)
- Yunsong Peng
- Division of Life Sciences and Medicine, School of Biomedical Engineering (Suzhou), University of Science and Technology of China, Hefei, China
- Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China
| | - Ziliang Cheng
- Department of Radiology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, China
| | - Chang Gong
- Guangdong Provincial Key Laboratory of Malignant Tumor Epigenetics and Gene Regulation, Breast Tumor Center, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, China
| | - Chushan Zheng
- Department of Radiology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, China
| | - Xiang Zhang
- Department of Radiology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, China
| | - Zhuo Wu
- Department of Radiology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, China
| | - Yaping Yang
- Guangdong Provincial Key Laboratory of Malignant Tumor Epigenetics and Gene Regulation, Breast Tumor Center, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, China
| | - Xiaodong Yang
- Division of Life Sciences and Medicine, School of Biomedical Engineering (Suzhou), University of Science and Technology of China, Hefei, China
- Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China
| | - Jian Zheng
- Division of Life Sciences and Medicine, School of Biomedical Engineering (Suzhou), University of Science and Technology of China, Hefei, China
- Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China
- *Correspondence: Jian Zheng, ; Jun Shen,
| | - Jun Shen
- Department of Radiology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, China
- *Correspondence: Jian Zheng, ; Jun Shen,
| |
Collapse
|
31
|
Tomita H, Kobayashi T, Takaya E, Mishiro S, Hirahara D, Fujikawa A, Kurihara Y, Mimura H, Kobayashi Y. Deep learning approach of diffusion-weighted imaging as an outcome predictor in laryngeal and hypopharyngeal cancer patients with radiotherapy-related curative treatment: a preliminary study. Eur Radiol 2022; 32:5353-5361. [PMID: 35201406 DOI: 10.1007/s00330-022-08630-9] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2021] [Revised: 01/15/2022] [Accepted: 02/02/2022] [Indexed: 11/04/2022]
Abstract
OBJECTIVES This preliminary study aimed to develop a deep learning (DL) model using diffusion-weighted imaging (DWI) and apparent diffusion coefficient (ADC) maps to predict local recurrence and 2-year progression-free survival (PFS) in laryngeal and hypopharyngeal cancer patients treated with various forms of radiotherapy-related curative therapy. METHODS Seventy patients with laryngeal and hypopharyngeal cancers treated by radiotherapy, chemoradiotherapy, or induction-(chemo)radiotherapy were enrolled and divided into training (N = 49) and test (N = 21) groups based on presentation timeline. All patients underwent MR before and 4 weeks after the start of radiotherapy. The DL models that extracted imaging features on pre- and intra-treatment DWI and ADC maps were trained to predict the local recurrence within a 2-year follow-up. In the test group, each DL model was analyzed for recurrence prediction. Additionally, the Kaplan-Meier and multivariable Cox regression analyses were performed to evaluate the prognostic significance of the DL models and clinical variables. RESULTS The highest area under the receiver operating characteristics curve and accuracy for predicting the local recurrence in the DL model were 0.767 and 81.0%, respectively, using intra-treatment DWI (DWIintra). The log-rank test showed that DWIintra was significantly associated with PFS (p = 0.013). DWIintra was an independent prognostic factor for PFS in multivariate analysis (p = 0.023). CONCLUSION DL models using DWIintra may have prognostic value in patients with laryngeal and hypopharyngeal cancers treated by curative radiotherapy. The model-related findings may contribute to determining the therapeutic strategy in the early stage of the treatment. KEY POINTS • Deep learning models using intra-treatment diffusion-weighted imaging have prognostic value in patients with laryngeal and hypopharyngeal cancers treated by curative radiotherapy. • The findings from these models may contribute to determining the therapeutic strategy at the early stage of the treatment.
Collapse
Affiliation(s)
- Hayato Tomita
- Department of Radiology, St. Marianna University School of Medicine, 2-16-1 Sugao, Miyamae-ku, Kawasaki, Kanagawa, 216-8511, Japan.
| | - Tatsuaki Kobayashi
- Department of Advanced Biomedical Imaging Informatics, St. Marianna University School of Medicine, 2-16-1 Sugao, Miyamae-ku, Kawasaki, Kanagawa, 216-8511, Japan
| | - Eichi Takaya
- School of Science for Open and Environmental Systems, Graduate School of Science and Technology, Keio University, 3-14-1 Hiyoshi, Kohoku-ku, Yokohama, Kanagawa, 223-8522, Japan
| | - Sono Mishiro
- Department of AI Research Lab, Harada Academy, 2-54-4, Higashitaniyama, Kagoshima, Kagoshima, 891-0113, Japan
| | - Daisuke Hirahara
- Department of AI Research Lab, Harada Academy, 2-54-4, Higashitaniyama, Kagoshima, Kagoshima, 891-0113, Japan
| | - Atsuko Fujikawa
- Department of Radiology, St. Marianna University School of Medicine, 2-16-1 Sugao, Miyamae-ku, Kawasaki, Kanagawa, 216-8511, Japan
| | - Yoshiko Kurihara
- Department of Radiology, Machida Municipal Hospital, 2-15-41 Asahi-cho, Machida, Tokyo, 194-0023, Japan
| | - Hidefumi Mimura
- Department of Radiology, St. Marianna University School of Medicine, 2-16-1 Sugao, Miyamae-ku, Kawasaki, Kanagawa, 216-8511, Japan
| | - Yasuyuki Kobayashi
- Department of Advanced Biomedical Imaging Informatics, St. Marianna University School of Medicine, 2-16-1 Sugao, Miyamae-ku, Kawasaki, Kanagawa, 216-8511, Japan
| |
Collapse
|
32
|
Basmadjian RB, Kong S, Boyne DJ, Jarada TN, Xu Y, Cheung WY, Lupichuk S, Quan ML, Brenner DR. Developing a Prediction Model for Pathologic Complete Response Following Neoadjuvant Chemotherapy in Breast Cancer: A Comparison of Model Building Approaches. JCO Clin Cancer Inform 2022; 6:e2100055. [PMID: 35148170 PMCID: PMC8846388 DOI: 10.1200/cci.21.00055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022] Open
Abstract
The optimal characteristics among patients with breast cancer to recommend neoadjuvant chemotherapy is an active area of clinical research. We developed and compared several approaches to developing prediction models for pathologic complete response (pCR) among patients with breast cancer in Alberta.
Collapse
Affiliation(s)
- Robert B Basmadjian
- Department of Community Health Sciences, Foothills Medical Centre, University of Calgary, Calgary, Alberta, Canada
| | - Shiying Kong
- Department of Community Health Sciences, Foothills Medical Centre, University of Calgary, Calgary, Alberta, Canada.,Department of Oncology, University of Calgary, Tom Baker Cancer Centre, Calgary, Alberta, Canada.,Department of Surgery, Foothills Medical Centre, University of Calgary, Calgary, Alberta, Canada
| | - Devon J Boyne
- Department of Community Health Sciences, Foothills Medical Centre, University of Calgary, Calgary, Alberta, Canada.,Department of Oncology, University of Calgary, Tom Baker Cancer Centre, Calgary, Alberta, Canada
| | - Tamer N Jarada
- Department of Oncology, University of Calgary, Tom Baker Cancer Centre, Calgary, Alberta, Canada
| | - Yuan Xu
- Department of Community Health Sciences, Foothills Medical Centre, University of Calgary, Calgary, Alberta, Canada.,Department of Oncology, University of Calgary, Tom Baker Cancer Centre, Calgary, Alberta, Canada.,Department of Surgery, Foothills Medical Centre, University of Calgary, Calgary, Alberta, Canada
| | - Winson Y Cheung
- Department of Community Health Sciences, Foothills Medical Centre, University of Calgary, Calgary, Alberta, Canada.,Department of Oncology, University of Calgary, Tom Baker Cancer Centre, Calgary, Alberta, Canada
| | - Sasha Lupichuk
- Department of Community Health Sciences, Foothills Medical Centre, University of Calgary, Calgary, Alberta, Canada.,Department of Oncology, University of Calgary, Tom Baker Cancer Centre, Calgary, Alberta, Canada
| | - May Lynn Quan
- Department of Community Health Sciences, Foothills Medical Centre, University of Calgary, Calgary, Alberta, Canada.,Department of Oncology, University of Calgary, Tom Baker Cancer Centre, Calgary, Alberta, Canada.,Department of Surgery, Foothills Medical Centre, University of Calgary, Calgary, Alberta, Canada
| | - Darren R Brenner
- Department of Community Health Sciences, Foothills Medical Centre, University of Calgary, Calgary, Alberta, Canada.,Department of Oncology, University of Calgary, Tom Baker Cancer Centre, Calgary, Alberta, Canada
| |
Collapse
|
33
|
Deep learning of quantitative ultrasound multi-parametric images at pre-treatment to predict breast cancer response to chemotherapy. Sci Rep 2022; 12:2244. [PMID: 35145158 PMCID: PMC8831592 DOI: 10.1038/s41598-022-06100-2] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2021] [Accepted: 01/20/2022] [Indexed: 12/24/2022] Open
Abstract
In this study, a novel deep learning-based methodology was investigated to predict breast cancer response to neo-adjuvant chemotherapy (NAC) using the quantitative ultrasound (QUS) multi-parametric imaging at pre-treatment. QUS multi-parametric images of breast tumors were generated using the data acquired from 181 patients diagnosed with locally advanced breast cancer and planned for NAC followed by surgery. The ground truth response to NAC was identified for each patient after the surgery using the standard clinical and pathological criteria. Two deep convolutional neural network (DCNN) architectures including the residual network and residual attention network (RAN) were explored for extracting optimal feature maps from the parametric images, with a fully connected network for response prediction. In different experiments, the features maps were derived from the tumor core only, as well as the core and its margin. Evaluation results on an independent test set demonstrate that the developed model with the RAN architecture to extract feature maps from the expanded parametric images of the tumor core and margin had the best performance in response prediction with an accuracy of 88% and an area under the receiver operating characteristic curve of 0.86. Ten-year survival analyses indicate statistically significant differences between the survival of the responders and non-responders identified based on the model prediction at pre-treatment and the standard criteria at post-treatment. The results of this study demonstrate the promising capability of DCNNs with attention mechanisms in predicting breast cancer response to NAC prior to the start of treatment using QUS multi-parametric images.
Collapse
|
34
|
Mao N, Shi Y, Lian C, Wang Z, Zhang K, Xie H, Zhang H, Chen Q, Cheng G, Xu C, Dai Y. Intratumoral and peritumoral radiomics for preoperative prediction of neoadjuvant chemotherapy effect in breast cancer based on contrast-enhanced spectral mammography. Eur Radiol 2022; 32:3207-3219. [DOI: 10.1007/s00330-021-08414-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2021] [Revised: 09/26/2021] [Accepted: 10/13/2021] [Indexed: 12/14/2022]
|
35
|
Yan S, Peng H, Yu Q, Chen X, Liu Y, Zhu Y, Chen K, Wang P, Li Y, Zhang X, Meng W. Computer-aided classification of MRI for pathological complete response to neoadjuvant chemotherapy in breast cancer. Future Oncol 2021; 18:991-1001. [PMID: 34894719 DOI: 10.2217/fon-2021-1212] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
Background: To determine suitable optimal classifiers and examine the general applicability of computer-aided classification to compare the differences between a computer-aided system and radiologists in predicting pathological complete response (pCR) from patients with breast cancer receiving neoadjuvant chemotherapy. Methods: We analyzed a total of 455 masses and used the U-Net network and ResNet to execute MRI segmentation and pCR classification. The diagnostic performance of radiologists, the computer-aided system and a combination of radiologists and computer-aided system were compared using receiver operating characteristic curve analysis. Results: The combination of radiologists and computer-aided system had the best performance for predicting pCR with an area under the curve (AUC) value of 0.899, significantly higher than that of radiologists alone (AUC: 0.700) and computer-aided system alone (AUC: 0.835). Conclusion: An automated classification system is feasible to predict the pCR to neoadjuvant chemotherapy in patients with breast cancer and can complement MRI.
Collapse
Affiliation(s)
- Shaolei Yan
- Radiology Department, Harbin Medical University, Harbin Medical University Cancer Hospital, 150 Haping Road, Harbin, Heilongjiang, 150081, China
| | - Haiyong Peng
- Radiology Department, Harbin Medical University, Harbin Medical University Cancer Hospital, 150 Haping Road, Harbin, Heilongjiang, 150081, China
| | - Qiujie Yu
- Radiology Department, Harbin Medical University, Harbin Medical University Cancer Hospital, 150 Haping Road, Harbin, Heilongjiang, 150081, China
| | - Xiaodan Chen
- Department of Computer Technology, Harbin Institute of Technology University, 92 West Street, Harbin, Heilongjiang, 150000, China
| | - Yue Liu
- Department of Radiology, Dongzhimen Hospital, Beijing University of Chinese Medicine, No. 5, Haiyuncang, Dongcheng District, Beijing, 100700, China
| | - Ye Zhu
- Department of Obstetrics & Gynecology, Peking University People's Hospital, No. 11 Xizhimen South Street, Xicheng District, Beijing, 100044, China
| | - Kaige Chen
- Radiology Department, Harbin Medical University, Harbin Medical University Cancer Hospital, 150 Haping Road, Harbin, Heilongjiang, 150081, China
| | - Ping Wang
- Radiology Department, Harbin Medical University, Harbin Medical University Cancer Hospital, 150 Haping Road, Harbin, Heilongjiang, 150081, China
| | - Yujiao Li
- Radiology Department, Harbin Medical University, Harbin Medical University Cancer Hospital, 150 Haping Road, Harbin, Heilongjiang, 150081, China
| | - Xiushi Zhang
- Radiology Department, Harbin Medical University, Harbin Medical University Cancer Hospital, 150 Haping Road, Harbin, Heilongjiang, 150081, China
| | - Wei Meng
- Radiology Department, Harbin Medical University, Harbin Medical University Cancer Hospital, 150 Haping Road, Harbin, Heilongjiang, 150081, China
| |
Collapse
|
36
|
Svecic A, Mansour R, Tang A, Kadoury S. Prediction of post transarterial chemoembolization MR images of hepatocellular carcinoma using spatio-temporal graph convolutional networks. PLoS One 2021; 16:e0259692. [PMID: 34874934 PMCID: PMC8651128 DOI: 10.1371/journal.pone.0259692] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2021] [Accepted: 10/24/2021] [Indexed: 11/25/2022] Open
Abstract
Magnetic resonance imaging (MRI) plays a critical role in the planning and monitoring of hepatocellular carcinomas (HCC) treated with locoregional therapies, in order to assess disease progression or recurrence. Dynamic contrast-enhanced (DCE)-MRI sequences offer temporal data on tumor enhancement characteristics which has strong prognostic value. Yet, predicting follow-up DCE-MR images from which tumor enhancement and viability can be measured, before treatment of HCC actually begins, remains an unsolved problem given the complexity of spatial and temporal information. We propose an approach to predict future DCE-MRI examinations following transarterial chemoembolization (TACE) by learning the spatio-temporal features related to HCC response from pre-TACE images. A novel Spatial-Temporal Discriminant Graph Neural Network (STDGNN) based on graph convolutional networks is presented. First, embeddings of viable, equivocal and non-viable HCCs are separated within a joint low-dimensional latent space, which is created using a discriminant neural network representing tumor-specific features. Spatial tumoral features from independent MRI volumes are then extracted with a structural branch, while dynamic features are extracted from the multi-phase sequence with a separate temporal branch. The model extracts spatio-temporal features by a joint minimization of the network branches. At testing, a pre-TACE diagnostic DCE-MRI is embedded on the discriminant spatio-temporal latent space, which is then translated to the follow-up domain space, thus allowing to predict the post-TACE DCE-MRI describing HCC treatment response. A dataset of 366 HCC’s from liver cancer patients was used to train and test the model using DCE-MRI examinations with associated pathological outcomes, with the spatio-temporal framework yielding 93.5% classification accuracy in response identification, and generating follow-up images yielding insignificant differences in perfusion parameters compared to ground-truth post-TACE examinations.
Collapse
Affiliation(s)
- Andrei Svecic
- Department of Computer Engineering, MedICAL, Polytechnique Montréal, Montréal, Québec, Canada
| | | | - An Tang
- CHUM Research Center, Montréal, Québec, Canada
- Department of Radiology, CHUM, Montréal, Québec, Canada
| | - Samuel Kadoury
- Department of Computer Engineering, MedICAL, Polytechnique Montréal, Montréal, Québec, Canada
- CHUM Research Center, Montréal, Québec, Canada
- * E-mail:
| |
Collapse
|
37
|
Yousif M, van Diest PJ, Laurinavicius A, Rimm D, van der Laak J, Madabhushi A, Schnitt S, Pantanowitz L. Artificial intelligence applied to breast pathology. Virchows Arch 2021; 480:191-209. [PMID: 34791536 DOI: 10.1007/s00428-021-03213-3] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Revised: 09/12/2021] [Accepted: 09/27/2021] [Indexed: 12/12/2022]
Abstract
The convergence of digital pathology and computer vision is increasingly enabling computers to perform tasks performed by humans. As a result, artificial intelligence (AI) is having an astoundingly positive effect on the field of pathology, including breast pathology. Research using machine learning and the development of algorithms that learn patterns from labeled digital data based on "deep learning" neural networks and feature-engineered approaches to analyze histology images have recently provided promising results. Thus far, image analysis and more complex AI-based tools have demonstrated excellent success performing tasks such as the quantification of breast biomarkers and Ki67, mitosis detection, lymph node metastasis recognition, tissue segmentation for diagnosing breast carcinoma, prognostication, computational assessment of tumor-infiltrating lymphocytes, and prediction of molecular expression as well as treatment response and benefit of therapy from routine H&E images. This review critically examines the literature regarding these applications of AI in the area of breast pathology.
Collapse
Affiliation(s)
- Mustafa Yousif
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA.
- Department of Pathology, Vanderbilt University Medical Center, Nashville, TN, USA.
| | - Paul J van Diest
- Department of Pathology, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Arvydas Laurinavicius
- Department of Pathology, Pharmacology and Forensic Medicine, Faculty of Medicine, Vilnius University, and National Center of Pathology, Affiliate of Vilnius University Hospital Santaros Klinikos, Vilnius, Lithuania
| | - David Rimm
- Department of Pathology, Yale University School of Medicine, New Haven, CT, USA
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Anant Madabhushi
- Department of Biomedical Engineering, Case Western Reserve University, and Louis Stokes Cleveland Veterans Administration Medical Center, Cleveland, OH, USA
- Louis Stokes Cleveland Veterans Administration Medical Center, Cleveland, OH, USA
| | - Stuart Schnitt
- Department of Pathology, Brigham and Women's Hospital and Harvard Medical School, Breast Oncology Program, Dana-Farber/Brigham and Women's Cancer Center, Boston, MA, USA
| | | |
Collapse
|
38
|
Satake H, Ishigaki S, Ito R, Naganawa S. Radiomics in breast MRI: current progress toward clinical application in the era of artificial intelligence. Radiol Med 2021; 127:39-56. [PMID: 34704213 DOI: 10.1007/s11547-021-01423-y] [Citation(s) in RCA: 41] [Impact Index Per Article: 13.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2021] [Accepted: 10/14/2021] [Indexed: 12/11/2022]
Abstract
Breast magnetic resonance imaging (MRI) is the most sensitive imaging modality for breast cancer diagnosis and is widely used clinically. Dynamic contrast-enhanced MRI is the basis for breast MRI, but ultrafast images, T2-weighted images, and diffusion-weighted images are also taken to improve the characteristics of the lesion. Such multiparametric MRI with numerous morphological and functional data poses new challenges to radiologists, and thus, new tools for reliable, reproducible, and high-volume quantitative assessments are warranted. In this context, radiomics, which is an emerging field of research involving the conversion of digital medical images into mineable data for clinical decision-making and outcome prediction, has been gaining ground in oncology. Recent development in artificial intelligence has promoted radiomics studies in various fields including breast cancer treatment and numerous studies have been conducted. However, radiomics has shown a translational gap in clinical practice, and many issues remain to be solved. In this review, we will outline the steps of radiomics workflow and investigate clinical application of radiomics focusing on breast MRI based on published literature, as well as current discussion about limitations and challenges in radiomics.
Collapse
Affiliation(s)
- Hiroko Satake
- Department of Radiology, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan.
| | - Satoko Ishigaki
- Department of Radiology, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan
| | - Rintaro Ito
- Department of Radiology, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan
| | - Shinji Naganawa
- Department of Radiology, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8550, Japan
| |
Collapse
|
39
|
Sun R, Hou X, Li X, Xie Y, Nie S. Transfer Learning Strategy Based on Unsupervised Learning and Ensemble Learning for Breast Cancer Molecular Subtype Prediction Using Dynamic Contrast-Enhanced MRI. J Magn Reson Imaging 2021; 55:1518-1534. [PMID: 34668601 DOI: 10.1002/jmri.27955] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2021] [Revised: 09/27/2021] [Accepted: 10/01/2021] [Indexed: 01/10/2023] Open
Abstract
BACKGROUND Imaging-driven deep learning strategies focus on training from scratch and transfer learning. However, the performance of training from scratch is often impeded by the lack of large-scale labeled training data. Additionally, owing to the differences between source and target domains, analyzing medical image tasks satisfactorily via transfer learning based on ImageNet is difficult. PURPOSE To investigate two transfer learning algorithms for breast cancer molecular subtype prediction (luminal and non-luminal) based on unsupervised pre-training and ensemble learning: M_EL and B_EL, using malignant and benign datasets as the source domain, respectively. STUDY TYPE Retrospective. POPULATION Eight hundred and thirty-three female patients with histologically confirmed breast lesions (567 benign and 266 malignant cases) were selected. In the 5-fold cross-validation, the malignant cohort was randomly divided into 5 subsets to form a training set (80%) and a validation set (20%). FIELD STRENGTH/SEQUENCE 3.0 T, dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) using T1-weighted high-resolution isotropic volume examination. ASSESSMENT First, three datasets acquired at different times post-contrast were preprocessed as unlabeled source domains. Second, three baseline networks corresponding to the different MRI post-contrast phases were built, optimized by a combination of mutual information maximization between high- and low-level representations and prior distribution constraints. Next, the pre-trained networks were fine-tuned on the labeled target domain. Finally, prediction results were integrated using weighted voting-based ensemble learning. STATISTICAL TESTS Mean accuracy, precision, specificity, and area under receiver operating characteristic curve (AUC) were obtained with 5-fold cross-validation. P < 0.05 was considered to be statistically significant. RESULTS Compared with a convolutional long short-term memory network, pre-trained VGG-16, VGG-19, and DenseNet-121 from ImageNet, M_EL and B_EL exhibited significantly more optimized prediction performance (specificity: 90.5% and 89.9%; accuracy: 82.6% and 81.1%; precision: 91.2% and 90.9%; AUC: 0.836 and 0.823, respectively). DATA CONCLUSION Transfer learning based on unsupervised pre-training may facilitate automatic prediction of breast cancer molecular subtypes. LEVEL OF EVIDENCE 3 TECHNICAL EFFICACY: Stage 2.
Collapse
Affiliation(s)
- Rong Sun
- School of Medical Instrument and Food Engineering, University of Shanghai for Science and Technology, Shanghai, China
| | - Xuewen Hou
- School of Medical Instrument and Food Engineering, University of Shanghai for Science and Technology, Shanghai, China
| | - Xiujuan Li
- Medical Imaging Center, Tai'an Central Hospital, Tai'an, China
| | - Yuanzhong Xie
- Medical Imaging Center, Tai'an Central Hospital, Tai'an, China
| | - Shengdong Nie
- School of Medical Instrument and Food Engineering, University of Shanghai for Science and Technology, Shanghai, China
| |
Collapse
|
40
|
Skarping I, Larsson M, Förnvik D. Analysis of mammograms using artificial intelligence to predict response to neoadjuvant chemotherapy in breast cancer patients: proof of concept. Eur Radiol 2021; 32:3131-3141. [PMID: 34652522 PMCID: PMC9038782 DOI: 10.1007/s00330-021-08306-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2021] [Revised: 08/28/2021] [Accepted: 09/02/2021] [Indexed: 12/22/2022]
Abstract
Objectives In this proof of concept study, a deep learning–based method for automatic analysis of digital mammograms (DM) as a tool to aid in assessment of neoadjuvant chemotherapy (NACT) treatment response in breast cancer (BC) was examined. Methods Baseline DM from 453 patients receiving NACT between 2005 and 2019 were included in the study cohort. A deep learning system, using the aforementioned baseline DM, was developed to predict pathological complete response (pCR) in the surgical specimen after completion of NACT. Two image patches, one extracted around the detected tumour and the other from the corresponding position in the reference image, were fed into a classification network. For training and validation, 1485 images obtained from 400 patients were used, and the model was ultimately applied to a test set consisting of 53 patients. Results A total of 95 patients (21%) achieved pCR. The median patient age was 52.5 years (interquartile range 43.7–62.1), and 255 (56%) were premenopausal. The artificial intelligence (AI) model predicted the pCR as represented by the area under the curve of 0.71 (95% confidence interval 0.53–0.90; p = 0.035). The sensitivity was 46% at a fixed specificity of 90%. Conclusions Our study describes an AI platform using baseline DM to predict BC patients’ responses to NACT. The initial AI performance indicated the potential to aid in clinical decision-making. In order to continue exploring the clinical utility of AI in predicting responses to NACT for BC, further research, including refining the methodology and a larger sample size, is warranted. Key Points • We aimed to answer the following question: Prior to initiation of neoadjuvant chemotherapy, can artificial intelligence (AI) applied to digital mammograms (DM) predict breast tumour response? • DMs contain information that AI can make use of for predicting pathological complete (pCR) response after neoadjuvant chemotherapy for breast cancer. • By developing an AI system designed to focus on relevant parts of the DM, fully automatic pCR prediction can be done well enough to potentially aid in clinical decision-making.
Collapse
Affiliation(s)
- I Skarping
- Division of Oncology, Department of Clinical Sciences, Lund University, Lund, Sweden.
- Department of Clinical Physiology and Nuclear Medicine, Skane University Hospital, Lund, Sweden.
| | | | - D Förnvik
- Medical Radiation Physics, Department of Translational Medicine, Lund University, Skane University Hospital, Malmö, Sweden
| |
Collapse
|
41
|
Duanmu H, Bhattarai S, Li H, Cheng CC, Wang F, Teodoro G, Janssen EAM, Gogineni K, Subhedar P, Aneja R, Kong J. Spatial Attention-Based Deep Learning System for Breast Cancer Pathological Complete Response Prediction with Serial Histopathology Images in Multiple Stains. MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION : MICCAI ... INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION 2021; 12908:550-560. [PMID: 36222817 PMCID: PMC9535677 DOI: 10.1007/978-3-030-87237-3_53] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
In triple negative breast cancer (TNBC) treatment, early prediction of pathological complete response (PCR) from chemotherapy before surgical operations is crucial for optimal treatment planning. We propose a novel deep learning-based system to predict PCR to neoadjuvant chemotherapy for TNBC patients with multi-stained histopathology images of serial tissue sections. By first performing tumor cell detection and recognition in a cell detection module, we produce a set of feature maps that capture cell type, shape, and location information. Next, a newly designed spatial attention module integrates such feature maps with original pathology images in multiple stains for enhanced PCR prediction in a dedicated prediction module. We compare it with baseline models that either use a single-stained slide or have no spatial attention module in place. Our proposed system yields 78.3% and 87.5% of accuracy for patch-, and patient-level PCR prediction, respectively, outperforming all other baseline models. Additionally, the heatmaps generated from the spatial attention module can help pathologists in targeting tissue regions important for disease assessment. Our system presents high efficiency and effectiveness and improves interpretability, making it highly promising for immediate clinical and translational impact.
Collapse
Affiliation(s)
| | | | - Hongxiao Li
- Georgia State University, Atlanta, GA 30302, USA
| | | | - Fusheng Wang
- Stony Brook University, Stony Brook, NY 11794, USA
| | - George Teodoro
- Federal University of Minas Gerais, Belo Horizonte 31270-010, Brazil
| | - Emiel A M Janssen
- Department of Pathology, Stavanger University Hospital, Stavanger, Norway
| | | | | | - Ritu Aneja
- Georgia State University, Atlanta, GA 30302, USA
| | - Jun Kong
- Georgia State University, Atlanta, GA 30302, USA
- Emory University, Atlanta, GA 30322, USA
| |
Collapse
|
42
|
Hickman SE, Baxter GC, Gilbert FJ. Adoption of artificial intelligence in breast imaging: evaluation, ethical constraints and limitations. Br J Cancer 2021; 125:15-22. [PMID: 33772149 PMCID: PMC8257639 DOI: 10.1038/s41416-021-01333-w] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2020] [Revised: 02/15/2021] [Accepted: 02/24/2021] [Indexed: 02/07/2023] Open
Abstract
Retrospective studies have shown artificial intelligence (AI) algorithms can match as well as enhance radiologist's performance in breast screening. These tools can facilitate tasks not feasible by humans such as the automatic triage of patients and prediction of treatment outcomes. Breast imaging faces growing pressure with the exponential growth in imaging requests and a predicted reduced workforce to provide reports. Solutions to alleviate these pressures are being sought with an increasing interest in the adoption of AI to improve workflow efficiency as well as patient outcomes. Vast quantities of data are needed to test and monitor AI algorithms before and after their incorporation into healthcare systems. Availability of data is currently limited, although strategies are being devised to harness the data that already exists within healthcare institutions. Challenges that underpin the realisation of AI into everyday breast imaging cannot be underestimated and the provision of guidance from national agencies to tackle these challenges, taking into account views from a societal, industrial and healthcare prospective is essential. This review provides background on the evaluation and use of AI in breast imaging in addition to exploring key ethical, technical, legal and regulatory challenges that have been identified so far.
Collapse
Affiliation(s)
- Sarah E Hickman
- Department of Radiology, University of Cambridge School of Clinical Medicine, Cambridge, UK
| | - Gabrielle C Baxter
- Department of Radiology, University of Cambridge School of Clinical Medicine, Cambridge, UK
| | - Fiona J Gilbert
- Department of Radiology, University of Cambridge School of Clinical Medicine, Cambridge, UK.
- Department of Radiology, Addenbrookes Hospital, Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK.
| |
Collapse
|
43
|
Hussain L, Huang P, Nguyen T, Lone KJ, Ali A, Khan MS, Li H, Suh DY, Duong TQ. Machine learning classification of texture features of MRI breast tumor and peri-tumor of combined pre- and early treatment predicts pathologic complete response. Biomed Eng Online 2021; 20:63. [PMID: 34183038 PMCID: PMC8240261 DOI: 10.1186/s12938-021-00899-z] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2021] [Accepted: 06/09/2021] [Indexed: 12/02/2022] Open
Abstract
Purpose This study used machine learning classification of texture features from MRI of breast tumor and peri-tumor at multiple treatment time points in conjunction with molecular subtypes to predict eventual pathological complete response (PCR) to neoadjuvant chemotherapy. Materials and method This study employed a subset of patients (N = 166) with PCR data from the I-SPY-1 TRIAL (2002–2006). This cohort consisted of patients with stage 2 or 3 breast cancer that underwent anthracycline–cyclophosphamide and taxane treatment. Magnetic resonance imaging (MRI) was acquired pre-neoadjuvant chemotherapy, early, and mid-treatment. Texture features were extracted from post-contrast-enhanced MRI, pre- and post-contrast subtraction images, and with morphological dilation to include peri-tumoral tissue. Molecular subtypes and Ki67 were also included in the prediction model. Performance of classification models used the receiver operating characteristics curve analysis including area under the curve (AUC). Statistical analysis was done using unpaired two-tailed t-tests. Results Molecular subtypes alone yielded moderate prediction performance of PCR (AUC = 0.82, p = 0.07). Pre-, early, and mid-treatment data alone yielded moderate performance (AUC = 0.88, 0.72, and 0.78, p = 0.03, 0.13, 0.44, respectively). The combined pre- and early treatment data markedly improved performance (AUC = 0.96, p = 0.0003). Addition of molecular subtypes improved performance slightly for individual time points but substantially for the combined pre- and early treatment (AUC = 0.98, p = 0.0003). The optimal morphological dilation was 3–5 pixels. Subtraction of post- and pre-contrast MRI further improved performance (AUC = 0.98, p = 0.00003). Finally, among the machine-learning algorithms evaluated, the RUSBoosted Tree machine-learning method yielded the highest performance. Conclusion AI-classification of texture features from MRI of breast tumor at multiple treatment time points accurately predicts eventual PCR. Longitudinal changes in texture features and peri-tumoral features further improve PCR prediction performance. Accurate assessment of treatment efficacy early on could minimize unnecessary toxic chemotherapy and enable mid-treatment modification for patients to achieve better clinical outcomes.
Collapse
Affiliation(s)
- Lal Hussain
- Department of Computer Science & IT, Neelum Campus, The University of Azad Jammu and Kashmir, Muzaffarabad, Azad Kashmir, Pakistan.,Department of Computer Science & IT, King Abdullah Campus, The University of Azad Jammu and Kashmir, Muzaffarabad, Azad Kashmir, Pakistan.,Department of Radiology, Renaissance School of Medicine At Stony, Brook University, 101 Nicolls Rd, Stony Brook, NY, 11794, USA.,Department of Radiology, Albert Einstein College of Medicine and Montefiore Medical Center, 111 East 210th Street, Bronx, NY, 10467, USA
| | - Pauline Huang
- Department of Radiology, Renaissance School of Medicine At Stony, Brook University, 101 Nicolls Rd, Stony Brook, NY, 11794, USA
| | - Tony Nguyen
- Department of Radiology, Renaissance School of Medicine At Stony, Brook University, 101 Nicolls Rd, Stony Brook, NY, 11794, USA
| | - Kashif J Lone
- Department of Computer Science & IT, King Abdullah Campus, The University of Azad Jammu and Kashmir, Muzaffarabad, Azad Kashmir, Pakistan
| | - Amjad Ali
- Department of Computer Science, COMSATS University Islamabad, Lahore Campus, Lahore, Pakistan
| | - Muhammad Salman Khan
- Department of Computer Science, COMSATS University Islamabad, Lahore Campus, Lahore, Pakistan
| | - Haifang Li
- Department of Radiology, Renaissance School of Medicine At Stony, Brook University, 101 Nicolls Rd, Stony Brook, NY, 11794, USA
| | - Doug Young Suh
- College of Electronics and Convergence Engineering, Kyung Hee University, Seoul, South Korea.
| | - Tim Q Duong
- Department of Radiology, Albert Einstein College of Medicine and Montefiore Medical Center, 111 East 210th Street, Bronx, NY, 10467, USA
| |
Collapse
|
44
|
Xiang H, Huang YS, Lee CH, Chang Chien TY, Lee CK, Liu L, Li A, Lin X, Chang RF. 3-D Res-CapsNet convolutional neural network on automated breast ultrasound tumor diagnosis. Eur J Radiol 2021; 138:109608. [PMID: 33711572 DOI: 10.1016/j.ejrad.2021.109608] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2020] [Revised: 02/06/2021] [Accepted: 02/19/2021] [Indexed: 12/24/2022]
Abstract
PURPOSE We propose a 3-D tumor computer-aided diagnosis (CADx) system with U-net and a residual-capsule neural network (Res-CapsNet) for ABUS images and provide a reference for early tumor diagnosis, especially non-mass lesions. METHODS A total of 396 patients with 444 tumors (226 malignant and 218 benign) were retrospectively enrolled from Sun Yat-sen University Cancer Center. In our CADx, preprocessing was performed first to crop and resize the tumor volumes of interest (VOIs). Then, a 3-D U-net and postprocessing were applied to the VOIs to obtain tumor masks. Finally, a 3-D Res-CapsNet classification model was executed with the VOIs and the corresponding masks to diagnose the tumors. Finally, the diagnostic performance, including accuracy, sensitivity, specificity, and area under the curve (AUC), was compared with other classification models and among three readers with different years of experience in ABUS review. RESULTS For all tumors, the accuracy, sensitivity, specificity, and AUC of the proposed CADx were 84.9 %, 87.2 %, 82.6 %, and 0.9122, respectively, outperforming other models and junior reader. Next, the tumors were subdivided into mass and non-mass tumors to validate the system performance. For mass tumors, our CADx achieved an accuracy, sensitivity, specificity, and AUC of 85.2 %, 88.2 %, 82.3 %, and 0.9147, respectively, which was higher than that of other models and junior reader. For non-mass tumors, our CADx achieved an accuracy, sensitivity, specificity, and AUC of 81.6 %, 78.3 %, 86.7 %, and 0.8654, respectively, outperforming the two readers. CONCLUSION The proposed CADx with 3-D U-net and 3-D Res-CapsNet models has the potential to reduce misdiagnosis, especially for non-mass lesions.
Collapse
Affiliation(s)
- Huiling Xiang
- Department of Ultrasound, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, China
| | - Yao-Sian Huang
- Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan
| | - Chu-Hsuan Lee
- Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan
| | - Ting-Yin Chang Chien
- Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan
| | | | - Lixian Liu
- Department of Ultrasound, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, China
| | - Anhua Li
- Department of Ultrasound, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, China
| | - Xi Lin
- Department of Ultrasound, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, China.
| | - Ruey-Feng Chang
- Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan; Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, Taipei, Taiwan.
| |
Collapse
|
45
|
Ou WC, Polat D, Dogan BE. Deep learning in breast radiology: current progress and future directions. Eur Radiol 2021; 31:4872-4885. [PMID: 33449174 DOI: 10.1007/s00330-020-07640-9] [Citation(s) in RCA: 27] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2020] [Revised: 10/30/2020] [Accepted: 12/17/2020] [Indexed: 12/13/2022]
Abstract
This review provides an overview of current applications of deep learning methods within breast radiology. The diagnostic capabilities of deep learning in breast radiology continue to improve, giving rise to the prospect that these methods may be integrated not only into detection and classification of breast lesions, but also into areas such as risk estimation and prediction of tumor responses to therapy. Remaining challenges include limited availability of high-quality data with expert annotations and ground truth determinations, the need for further validation of initial results, and unresolved medicolegal considerations. KEY POINTS: • Deep learning (DL) continues to push the boundaries of what can be accomplished by artificial intelligence (AI) in breast imaging with distinct advantages over conventional computer-aided detection. • DL-based AI has the potential to augment the capabilities of breast radiologists by improving diagnostic accuracy, increasing efficiency, and supporting clinical decision-making through prediction of prognosis and therapeutic response. • Remaining challenges to DL implementation include a paucity of prospective data on DL utilization and yet unresolved medicolegal questions regarding increasing AI utilization.
Collapse
Affiliation(s)
- William C Ou
- Department of Radiology, Seay Biomedical Building, University of Texas Southwestern Medical Center, 2201 Inwood Road, Dallas, TX, 75390, USA.
| | - Dogan Polat
- Department of Radiology, Seay Biomedical Building, University of Texas Southwestern Medical Center, 2201 Inwood Road, Dallas, TX, 75390, USA
| | - Basak E Dogan
- Department of Radiology, Seay Biomedical Building, University of Texas Southwestern Medical Center, 2201 Inwood Road, Dallas, TX, 75390, USA
| |
Collapse
|
46
|
Meyer-Bäse A, Morra L, Meyer-Bäse U, Pinker K. Current Status and Future Perspectives of Artificial Intelligence in Magnetic Resonance Breast Imaging. CONTRAST MEDIA & MOLECULAR IMAGING 2020; 2020:6805710. [PMID: 32934610 PMCID: PMC7474774 DOI: 10.1155/2020/6805710] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/09/2020] [Revised: 04/17/2020] [Accepted: 05/28/2020] [Indexed: 12/12/2022]
Abstract
Recent advances in artificial intelligence (AI) and deep learning (DL) have impacted many scientific fields including biomedical maging. Magnetic resonance imaging (MRI) is a well-established method in breast imaging with several indications including screening, staging, and therapy monitoring. The rapid development and subsequent implementation of AI into clinical breast MRI has the potential to affect clinical decision-making, guide treatment selection, and improve patient outcomes. The goal of this review is to provide a comprehensive picture of the current status and future perspectives of AI in breast MRI. We will review DL applications and compare them to standard data-driven techniques. We will emphasize the important aspect of developing quantitative imaging biomarkers for precision medicine and the potential of breast MRI and DL in this context. Finally, we will discuss future challenges of DL applications for breast MRI and an AI-augmented clinical decision strategy.
Collapse
Affiliation(s)
- Anke Meyer-Bäse
- Department of Scientific Computing, Florida State University, Tallahassee, Florida 32310-4120, USA
| | - Lia Morra
- Dipartimento di Automatica e Informatica, Politecnico di Torino, Torino, Italy
| | - Uwe Meyer-Bäse
- Department of Electrical and Computer Engineering, Florida A&M University and Florida State University, Tallahassee, Florida 32310-4120, USA
| | - Katja Pinker
- Department of Biomedical Imaging and Image-Guided Therapy, Division of Molecular and Gender Imaging, Medical University of Vienna, Vienna, Austria
- Department of Radiology, Memorial Sloan-Kettering Cancer Center, New York, New York 10065, USA
| |
Collapse
|