Breast Cancer Dataset, Classification and Detection Using Deep Learning

Multimodal data integration for oncology in the era of deep neural networks: a review

Cancer research encompasses data across various scales, modalities, and resolutions, from screening and diagnostic imaging to digitized histopathology slides to various types of molecular data and clinical records. The integration of these diverse data types for personalized cancer care and predictive modeling holds the promise of enhancing the accuracy and reliability of cancer screening, diagnosis, and treatment. Traditional analytical methods, which often focus on isolated or unimodal information, fall short of capturing the complex and heterogeneous nature of cancer data. The advent of deep neural networks has spurred the development of sophisticated multimodal data fusion techniques capable of extracting and synthesizing information from disparate sources. Among these, Graph Neural Networks (GNNs) and Transformers have emerged as powerful tools for multimodal learning, demonstrating significant success. This review presents the foundational principles of multimodal learning including oncology data modalities, taxonomy of multimodal learning, and fusion strategies. We delve into the recent advancements in GNNs and Transformers for the fusion of multimodal data in oncology, spotlighting key studies and their pivotal findings. We discuss the unique challenges of multimodal learning, such as data heterogeneity and integration complexities, alongside the opportunities it presents for a more nuanced and comprehensive understanding of cancer. Finally, we present some of the latest comprehensive multimodal pan-cancer data sources. By surveying the landscape of multimodal data integration in oncology, our goal is to underline the transformative potential of multimodal GNNs and Transformers. Through technological advancements and the methodological innovations presented in this review, we aim to chart a course for future research in this promising field. This review may be the first that highlights the current state of multimodal modeling applications in cancer using GNNs and transformers, presents comprehensive multimodal oncology data sources, and sets the stage for multimodal evolution, encouraging further exploration and development in personalized cancer care.

Current status and prospects of artificial intelligence in breast cancer pathology: convolutional neural networks to prospective Vision Transformers

Breast cancer is the most prevalent cancer among women, and its diagnosis requires the accurate identification and classification of histological features for effective patient management. Artificial intelligence, particularly through deep learning, represents the next frontier in cancer diagnosis and management. Notably, the use of convolutional neural networks and emerging Vision Transformers (ViT) has been reported to automate pathologists' tasks, including tumor detection and classification, in addition to improving the efficiency of pathology services. Deep learning applications have also been extended to the prediction of protein expression, molecular subtype, mutation status, therapeutic efficacy, and outcome prediction directly from hematoxylin and eosin-stained slides, bypassing the need for immunohistochemistry or genetic testing. This review explores the current status and prospects of deep learning in breast cancer diagnosis with a focus on whole-slide image analysis. Artificial intelligence applications are increasingly applied to many tasks in breast pathology ranging from disease diagnosis to outcome prediction, thus serving as valuable tools for assisting pathologists and supporting breast cancer management.

Multi-Level Swin Transformer Enabled Automatic Segmentation and Classification of Breast Metastases

Detection of metastatic breast cancer lesions is a challenging task in breast cancer treatment. The recent advancements in deep learning gained attention owing to its robustness, particularly in addressing automated segmentation and classification issues in medical images. In this paper, we proposed a modified Swin Transformer model (mST) integrated with a novel Multi-Level Adaptive Feature Fusion (MLAFF) Module. We constructed a modified Swin Transformer network comprising of a Local Transferable MSA (LT-MSA) and a Global Transferable MSA (GT-MSA) in addition to a Feed Forward Network (FFN). Our novel Multi-Level Adaptive Feature Fusion (MLAFF) module iteratively combines the features throughout multiple transformers. We utilized a pre-trained deep learning model U-Net and trained it on mammography utilizing Transfer Learning for automated segmentation. The proposed method, mST-MLAFF, is used for breast cancer classification into normal, benign, and malignant classes. Our model outperformed comparison methods based on U-Net and Swin Transformer in breast metastatic lesion segmentation on the seven benchmark datasets, namely INBreast, DDSM, MIAS, CBIS-DDSM, MIMBCD-UI, KAU-BCMD, and Mammographic Masses. Our model achieved 98% Dice-Similarity coefficient (DSC) for segmentation and an average of 94.5% accuracy for classification, whereas U-Net based model achieved 92% DSC and Swin Transformer achieved 93% DSC. Extensive performance evaluation of our model on benchmark datasets shows the potential of our model for breast cancer classification.Clinical relevance- This research work is focused on assisting the radiologist in the early detection and classification of breast cancer. A single mammography image is analyzed in less than a minute for automated segmentation and classification into malignant and benign classes.

Identification of Clinical Features Associated with Mortality in COVID-19 Patients

Understanding clinical features and risk factors associated with COVID-19 mortality is needed to early identify critically ill patients, initiate treatments and prevent mortality. A retrospective study on COVID-19 patients referred to a tertiary hospital in Iran between March and November 2020 was conducted. COVID-19-related mortality and its association with clinical features including headache, chest pain, symptoms on computerized tomography (CT), hospitalization, time to infection, history of neurological disorders, having a single or multiple risk factors, fever, myalgia, dizziness, seizure, abdominal pain, nausea, vomiting, diarrhoea and anorexia were investigated. Based on the investigation outcome, decision tree and dimension reduction algorithms were used to identify the aforementioned risk factors. Of the 3008 patients (mean age 59.3 ± 18.7 years, 44% women) with COVID-19, 373 died. There was a significant association between COVID-19 mortality and old age, headache, chest pain, low respiratory rate, oxygen saturation < 93%, need for a mechanical ventilator, having symptoms on CT, hospitalization, time to infection, neurological disorders, cardiovascular diseases and having a risk factor or multiple risk factors. In contrast, there was no significant association between mortality and gender, fever, myalgia, dizziness, seizure, abdominal pain, nausea, vomiting, diarrhoea and anorexia. Our results might help identify early symptoms related to COVID-19 and better manage patients according to the extracted decision tree. The proposed ML models identified a number of clinical features and risk factors associated with mortality in COVID-19 patients. These models if implemented in a clinical setting might help to early identify patients needing medical attention and care. However, more studies are needed to confirm these findings.

Applying Explainable Machine Learning Models for Detection of Breast Cancer Lymph Node Metastasis in Patients Eligible for Neoadjuvant Treatment

BACKGROUND

Due to recent changes in breast cancer treatment strategy, significantly more patients are treated with neoadjuvant systemic therapy (NST). Radiological methods do not precisely determine axillary lymph node status, with up to 30% of patients being misdiagnosed. Hence, supplementary methods for lymph node status assessment are needed. This study aimed to apply and evaluate machine learning models on clinicopathological data, with a focus on patients meeting NST criteria, for lymph node metastasis prediction.

METHODS

From the total breast cancer patient data (n = 8381), 719 patients were identified as eligible for NST. Machine learning models were applied for the NST-criteria group and the total study population. Model explainability was obtained by calculating Shapley values.

RESULTS

In the NST-criteria group, random forest achieved the highest performance (AUC: 0.793 [0.713, 0.865]), while in the total study population, XGBoost performed the best (AUC: 0.762 [0.726, 0.795]). Shapley values identified tumor size, Ki-67, and patient age as the most important predictors.

CONCLUSION

Tree-based models achieve a good performance in assessing lymph node status. Such models can lead to more accurate disease stage prediction and consecutively better treatment selection, especially for NST patients where radiological and clinical findings are often the only way of lymph node assessment.