Deep learning for identifying radiogenomic associations in breast cancer

Olecranon bone age assessment in puberty using a lateral elbow radiograph and a deep-learning model

OBJECTIVES

To improve pubertal bone age (BA) evaluation by developing a precise and practical elbow BA classification using the olecranon, and a deep-learning AI model.

MATERIALS AND METHODS

Lateral elbow radiographs taken for BA evaluation in children under 18 years were collected from January 2020 to June 2022, retrospectively. A novel classification and the olecranon BA were established based on the morphological changes in the olecranon ossification process during puberty. The olecranon BA was compared with other elbow and hand BA methods, using intraclass correlation coefficients (ICCs), and a deep-learning AI model was developed.

RESULTS

A total of 3508 lateral elbow radiographs (mean age 9.8 ± 1.8 years) were collected. The olecranon BA showed the highest applicability (100%) and interobserver agreement (ICC 0.993) among elbow BA methods. It showed excellent reliability with Sauvegrain (0.967 in girls, 0.969 in boys) and Dimeglio (0.978 in girls, 0.978 in boys) elbow BA methods, as well as Korean standard (KS) hand BA in boys (0.917), and good reliability with KS in girls (0.896) and Greulich-Pyle (GP)/Tanner-Whitehouse (TW)3 (0.835 in girls, 0.895 in boys) hand BA methods. The AI model for olecranon BA showed an accuracy of 0.96 and a specificity of 0.98 with EfficientDet-b4. External validation showed an accuracy of 0.86 and a specificity of 0.91.

CONCLUSION

The olecranon BA evaluation for puberty, requiring only a lateral elbow radiograph, showed the highest applicability and interobserver agreement, and excellent reliability with other BA evaluation methods, along with a high performance of the AI model.

CLINICAL RELEVANCE STATEMENT

This AI model uses a single lateral elbow radiograph to determine bone age for puberty from the olecranon ossification center and can improve pubertal bone age assessment with the highest applicability and excellent reliability compared to previous methods.

KEY POINTS

Elbow bone age is valuable for pubertal bone age assessment, but conventional methods have limitations. Olecranon bone age and its AI model showed high performances for pubertal bone age assessment. Olecranon bone age system is practical and accurate while requiring only a single lateral elbow radiograph.

Radiogenomics: bridging the gap between imaging and genomics for precision oncology

Genomics allows the tracing of origin and evolution of cancer at molecular scale and underpin modern cancer diagnosis and treatment systems. Yet, molecular biomarker-guided clinical decision-making encounters major challenges in the realm of individualized medicine, consisting of the invasiveness of procedures and the sampling errors due to high tumor heterogeneity. By contrast, medical imaging enables noninvasive and global characterization of tumors at a low cost. In recent years, radiomics has overcomes the limitations of human visual evaluation by high-throughput quantitative analysis, enabling the comprehensive utilization of the vast amount of information underlying radiological images. The cross-scale integration of radiomics and genomics (hereafter radiogenomics) has the enormous potential to enhance cancer decoding and act as a catalyst for digital precision medicine. Herein, we provide a comprehensive overview of the current framework and potential clinical applications of radiogenomics in patient care. We also highlight recent research advances to illustrate how radiogenomics can address common clinical problems in solid tumors such as breast cancer, lung cancer, and glioma. Finally, we analyze existing literature to outline challenges and propose solutions, while also identifying future research pathways. We believe that the perspectives shared in this survey will provide a valuable guide for researchers in the realm of radiogenomics aiming to advance precision oncology.

Functions and mechanisms of RNA m6A regulators in breast cancer (Review)

Breast cancer (BC) is a major malignant tumor in females and the incidence rate of BC has increased worldwide in recent years. N6‑methyladenosine (m6A) is a methylation modification that occurs extensively in eukaryotic RNA. The abnormal expression of m6A and related regulatory proteins can activate or inhibit certain signal pathways or oncogenes, thus affecting the proliferation, metastasis and prognosis of BC. Numerous studies have shown that m6A regulator disorder exists in BC, and this disorder can be reversed. Therefore, m6A is predicted as a potential therapeutic target for BC. However, the molecular mechanism of m6A RNA methylation regulating the occurrence and development of BC has not been comprehensively elucidated. In this review article, the functions of various m6A regulators and the specific mechanisms of certain regulators of the progress of BC were summarized. Furthermore, the dual role of RNA methylation in tumor progression was discussed, concluding that RNA methylation can not only lead to tumorigenesis but at times give rise to inhibition of tumor formation. In addition, further comprehensive analysis on mechanisms of m6A regulators in BC is conducive to screening effective potential targets and formulating targeted treatment strategies, which will provide new methods for the prevention and treatment of BC.

Identifying radiogenomic associations of breast cancer based on DCE-MRI by using Siamese Neural Network with manufacturer bias normalization

BACKGROUND AND PURPOSE

The immunohistochemical test (IHC) for Human Epidermal Growth Factor Receptor 2 (HER2) and hormone receptors (HR) provides prognostic information and guides treatment for patients with invasive breast cancer. The objective of this paper is to establish a non-invasive system for identifying HER2 and HR in breast cancer using dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI).

METHODS

In light of the absence of high-performance algorithms and external validation in previously published methods, this study utilizes 3D deep features and radiomics features to represent the information of the Region of Interest (ROI). A Siamese Neural Network was employed as the classifier, with 3D deep features and radiomics features serving as the network input. To neutralize manufacturer bias, a batch effect normalization method, ComBat, was introduced. To enhance the reliability of the study, two datasets, Predict Your Therapeutic Response with Imaging and moLecular Analysis (I-SPY 1) and I-SPY 2, were incorporated. I-SPY 2 was utilized for model training and validation, while I-SPY 1 was exclusively employed for external validation. Additionally, a breast tumor segmentation network was trained to improve radiomic feature extraction.

RESULTS

The results indicate that our approach achieved an average Area Under the Curve (AUC) of 0.632, with a Standard Error of the Mean (SEM) of 0.042 for HER2 prediction in the I-SPY 2 dataset. For HR prediction, our method attained an AUC of 0.635 (SEM 0.041), surpassing other published methods in the AUC metric. Moreover, the proposed method yielded competitive results in other metrics. In external validation using the I-SPY 1 dataset, our approach achieved an AUC of 0.567 (SEM 0.032) for HR prediction and 0.563 (SEM 0.033) for HER2 prediction.

CONCLUSION

This study proposes a non-invasive system for identifying HER2 and HR in breast cancer. Although the results do not conclusively demonstrate superiority in both tasks, they indicate that the proposed method achieved good performance and is a competitive classifier compared to other reference methods. Ablation studies demonstrate that both radiomics features and deep features for the Siamese Neural Network are beneficial for the model. The introduced manufacturer bias normalization method has been shown to enhance the method's performance. Furthermore, the external validation of the method enhances the reliability of this research. Source code, pre-trained segmentation network, Radiomics and deep features, data for statistical analysis, and Supporting Information of this article are online at: https://github.com/FORRESTHUACHEN/Siamese_Neural_Network_based_Brest_cancer_Radiogenomic.

MOB-CBAM: A dual-channel attention-based deep learning generalizable model for breast cancer molecular subtypes prediction using mammograms

BACKGROUND AND OBJECTIVE

Deep Learning models have emerged as a significant tool in generating efficient solutions for complex problems including cancer detection, as they can analyze large amounts of data with high efficiency and performance. Recent medical studies highlight the significance of molecular subtype detection in breast cancer, aiding the development of personalized treatment plans as different subtypes of cancer respond better to different therapies.

METHODS

In this work, we propose a novel lightweight dual-channel attention-based deep learning model MOB-CBAM that utilizes the backbone of MobileNet-V3 architecture with a Convolutional Block Attention Module to make highly accurate and precise predictions about breast cancer. We used the CMMD mammogram dataset to evaluate the proposed model in our study. Nine distinct data subsets were created from the original dataset to perform coarse and fine-grained predictions, enabling it to identify masses, calcifications, benign, malignant tumors and molecular subtypes of cancer, including Luminal A, Luminal B, HER-2 Positive, and Triple Negative. The pipeline incorporates several image pre-processing techniques, including filtering, enhancement, and normalization, for enhancing the model's generalization ability.

RESULTS

While identifying benign versus malignant tumors, i.e., coarse-grained classification, the MOB-CBAM model produced exceptional results with 99 % accuracy, precision, recall, and F1-score values of 0.99 and MCC of 0.98. In terms of fine-grained classification, the MOB-CBAM model has proven to be highly efficient in accurately identifying mass with (benign/malignant) and calcification with (benign/malignant) classification tasks with an impressive accuracy rate of 98 %. We have also cross-validated the efficiency of the proposed MOB-CBAM deep learning architecture on two datasets: MIAS and CBIS-DDSM. On the MIAS dataset, an accuracy of 97 % was reported for the task of classifying benign, malignant, and normal images, while on the CBIS-DDSM dataset, an accuracy of 98 % was achieved for the classification of mass with either benign or malignant, and calcification with benign and malignant tumors.

CONCLUSION

This study presents lightweight MOB-CBAM, a novel deep learning framework, to address breast cancer diagnosis and subtype prediction. The model's innovative incorporation of the CBAM enhances precise predictions. The extensive evaluation of the CMMD dataset and cross-validation on other datasets affirm the model's efficacy.

Sequence based model using deep neural network and hybrid features for identification of 5-hydroxymethylcytosine modification

RNA modifications are pivotal in the development of newly synthesized structures, showcasing a vast array of alterations across various RNA classes. Among these, 5-hydroxymethylcytosine (5HMC) stands out, playing a crucial role in gene regulation and epigenetic changes, yet its detection through conventional methods proves cumbersome and costly. To address this, we propose Deep5HMC, a robust learning model leveraging machine learning algorithms and discriminative feature extraction techniques for accurate 5HMC sample identification. Our approach integrates seven feature extraction methods and various machine learning algorithms, including Random Forest, Naive Bayes, Decision Tree, and Support Vector Machine. Through K-fold cross-validation, our model achieved a notable 84.07% accuracy rate, surpassing previous models by 7.59%, signifying its potential in early cancer and cardiovascular disease diagnosis. This study underscores the promise of Deep5HMC in offering insights for improved medical assessment and treatment protocols, marking a significant advancement in RNA modification analysis.

Use of artificial intelligence in breast surgery: a narrative review

Background and Objective

We have witnessed tremendous advances in artificial intelligence (AI) technologies. Breast surgery, a subspecialty of general surgery, has notably benefited from AI technologies. This review aims to evaluate how AI has been integrated into breast surgery practices, to assess its effectiveness in improving surgical outcomes and operational efficiency, and to identify potential areas for future research and application.

Methods

Two authors independently conducted a comprehensive search of PubMed, Google Scholar, EMBASE, and Cochrane CENTRAL databases from January 1, 1950, to September 4, 2023, employing keywords pertinent to AI in conjunction with breast surgery or cancer. The search focused on English language publications, where relevance was determined through meticulous screening of titles, abstracts, and full-texts, followed by an additional review of references within these articles. The review covered a range of studies illustrating the applications of AI in breast surgery encompassing lesion diagnosis to postoperative follow-up. Publications focusing specifically on breast reconstruction were excluded.

Key Content and Findings

AI models have preoperative, intraoperative, and postoperative applications in the field of breast surgery. Using breast imaging scans and patient data, AI models have been designed to predict the risk of breast cancer and determine the need for breast cancer surgery. In addition, using breast imaging scans and histopathological slides, models were used for detecting, classifying, segmenting, grading, and staging breast tumors. Preoperative applications included patient education and the display of expected aesthetic outcomes. Models were also designed to provide intraoperative assistance for precise tumor resection and margin status assessment. As well, AI was used to predict postoperative complications, survival, and cancer recurrence.

Conclusions

Extra research is required to move AI models from the experimental stage to actual implementation in healthcare. With the rapid evolution of AI, further applications are expected in the coming years including direct performance of breast surgery. Breast surgeons should be updated with the advances in AI applications in breast surgery to provide the best care for their patients.

The Convergence of Radiology and Genomics: Advancing Breast Cancer Diagnosis with Radiogenomics

Despite significant progress in the prevention, screening, diagnosis, prognosis, and therapy of breast cancer (BC), it remains a highly prevalent and life-threatening disease affecting millions worldwide. Molecular subtyping of BC is crucial for predictive and prognostic purposes due to the diverse clinical behaviors observed across various types. The molecular heterogeneity of BC poses uncertainties in its impact on diagnosis, prognosis, and treatment. Numerous studies have highlighted genetic and environmental differences between patients from different geographic regions, emphasizing the need for localized research. International studies have revealed that patients with African heritage are often diagnosed at a more advanced stage and exhibit poorer responses to treatment and lower survival rates. Despite these global findings, there is a dearth of in-depth studies focusing on communities in the African region. Early diagnosis and timely treatment are paramount to improving survival rates. In this context, radiogenomics emerges as a promising field within precision medicine. By associating genetic patterns with image attributes or features, radiogenomics has the potential to significantly improve early detection, prognosis, and diagnosis. It can provide valuable insights into potential treatment options and predict the likelihood of survival, progression, and relapse. Radiogenomics allows for visual features and genetic marker linkage that promises to eliminate the need for biopsy and sequencing. The application of radiogenomics not only contributes to advancing precision oncology and individualized patient treatment but also streamlines clinical workflows. This review aims to delve into the theoretical underpinnings of radiogenomics and explore its practical applications in the diagnosis, management, and treatment of BC and to put radiogenomics on a path towards fully integrated diagnostics.

Molecular-subtype guided automatic invasive breast cancer grading using dynamic contrast-enhanced MRI

BACKGROUND AND OBJECTIVES

Histological grade and molecular subtype have presented valuable references in assigning personalized or precision medicine as the significant prognostic indicators representing biological behaviors of invasive breast cancer (IBC). To evaluate a two-stage deep learning framework for IBC grading that incorporates with molecular-subtype (MS) information using DCE-MRI.

METHODS

In Stage I, an innovative neural network called IOS2-DA is developed, which includes a dense atrous-spatial pyramid pooling block with a pooling layer (DA) and inception-octconved blocks with double kernel squeeze-and-excitations (IOS2). This method focuses on the imaging manifestation of IBC grades and performs preliminary prediction using a novel class F1-score loss function. In Stage II, a MS attention branch is introduced to fine-tune the integrated deep vectors from IOS2-DA via Kullback-Leibler divergence. The MS-guided information is weighted with preliminary results to obtain classification values, which are analyzed by ensemble learning for tumor grade prediction on three MRI post-contrast series. Objective assessment is quantitatively evaluated by receiver operating characteristic curve analysis. DeLong test is applied to measure statistical significance (P < 0.05).

RESULTS

The molecular-subtype guided IOS2-DA performs significantly better than the single IOS2-DA in terms of accuracy (0.927), precision (0.942), AUC (0.927, 95% CI: [0.908, 0.946]), and F1-score (0.930). The gradient-weighted class activation maps show that the feature representations extracted from IOS2-DA are consistent with tumor areas.

CONCLUSIONS

IOS2-DA elucidates its potential in non-invasive tumor grade prediction. With respect to the correlation between MS and histological grade, it exhibits remarkable clinical prospects in the application of relevant clinical biomarkers to enhance the diagnostic effectiveness of IBC grading. Therefore, DCE-MRI tends to be a feasible imaging modality for the thorough preoperative assessment of breast biological behavior and carcinoma prognosis.

The use of radiomics in magnetic resonance imaging for the pre-treatment characterisation of breast cancers: A scoping review

Radiomics is an emerging field that aims to extract and analyse a comprehensive set of quantitative features from medical images. This scoping review is focused on MRI-based radiomic features for the molecular profiling of breast tumours and the implications of this work for predicting patient outcomes. A thorough systematic literature search and outcome extraction were performed to identify relevant studies published in MEDLINE/PubMed (National Centre for Biotechnology Information), EMBASE and Scopus from 2015 onwards. The following information was retrieved from each article: study purpose, study design, extracted radiomic features, machine learning technique(s), sample size/characteristics, statistical result(s) and implications on patient outcomes. Based on the study purpose, four key themes were identified in the included 63 studies: tumour subtype classification (n = 35), pathologically complete response (pCR) prediction (n = 15), lymph node metastasis (LNM) detection (n = 7) and recurrence rate prediction (n = 6). In all four themes, reported accuracies widely varied among the studies, for example, area under receiver characteristics curve (AUC) for detecting LNM ranged from 0.72 to 0.91 and the AUC for predicting pCR ranged from 0.71 to 0.99. In all four themes, combining radiomic features with clinical data improved the predictive models. Preliminary results of this study showed radiomics potential to characterise the whole tumour heterogeneity, with clear implications for individual-targeted treatment. However, radiomics is still in the pre-clinical phase, currently with an insufficient number of large multicentre studies and those existing studies are often limited by insufficient methodological transparency and standardised workflow. Consequently, the clinical translation of existing studies is currently limited.

Identification of Luminal A breast cancer by using deep learning analysis based on multi-modal images

Purpose

To evaluate the diagnostic performance of a deep learning model based on multi-modal images in identifying molecular subtype of breast cancer.

Materials and methods

A total of 158 breast cancer patients (170 lesions, median age, 50.8 ± 11.0 years), including 78 Luminal A subtype and 92 non-Luminal A subtype lesions, were retrospectively analyzed and divided into a training set (n = 100), test set (n = 45), and validation set (n = 25). Mammography (MG) and magnetic resonance imaging (MRI) images were used. Five single-mode models, i.e., MG, T2-weighted imaging (T2WI), diffusion weighting imaging (DWI), axial apparent dispersion coefficient (ADC), and dynamic contrast-enhanced MRI (DCE-MRI), were selected. The deep learning network ResNet50 was used as the basic feature extraction and classification network to construct the molecular subtype identification model. The receiver operating characteristic curve were used to evaluate the prediction efficiency of each model.

Results

The accuracy, sensitivity and specificity of a multi-modal tool for identifying Luminal A subtype were 0.711, 0.889, and 0.593, respectively, and the area under the curve (AUC) was 0.802 (95% CI, 0.657- 0.906); the accuracy, sensitivity, and AUC were higher than those of any single-modal model, but the specificity was slightly lower than that of DCE-MRI model. The AUC value of MG, T2WI, DWI, ADC, and DCE-MRI model was 0.593 (95%CI, 0.436-0.737), 0.700 (95%CI, 0.545-0.827), 0.564 (95%CI, 0.408-0.711), 0.679 (95%CI, 0.523-0.810), and 0.553 (95%CI, 0.398-0.702), respectively.

Conclusion

The combination of deep learning and multi-modal imaging is of great significance for diagnosing breast cancer subtypes and selecting personalized treatment plans for doctors.

Weakly Supervised Breast Lesion Detection in Dynamic Contrast-Enhanced MRI

Currently, obtaining accurate medical annotations requires high labor and time effort, which largely limits the development of supervised learning-based tumor detection tasks. In this work, we investigated a weakly supervised learning model for detecting breast lesions in dynamic contrast-enhanced MRI (DCE-MRI) with only image-level labels. Two hundred fifty-four normal and 398 abnormal cases with pathologically confirmed lesions were retrospectively enrolled into the breast dataset, which was divided into the training set (80%), validation set (10%), and testing set (10%) at the patient level. First, the second image series S2 after the injection of a contrast agent was acquired from the 3.0-T, T1-weighted dynamic enhanced MR imaging sequences. Second, a feature pyramid network (FPN) with convolutional block attention module (CBAM) was proposed to extract multi-scale feature maps of the modified classification network VGG16. Then, initial location information was obtained from the heatmaps generated using the layer class activation mapping algorithm (Layer-CAM). Finally, the detection results of breast lesion were refined by the conditional random field (CRF). Accuracy, sensitivity, specificity, and area under the receiver operating characteristic (ROC) curve (AUC) were utilized for evaluation of image-level classification. Average precision (AP) was estimated for breast lesion localization. Delong's test was used to compare the AUCs of different models for significance. The proposed model was effective with accuracy of 95.2%, sensitivity of 91.6%, specificity of 99.2%, and AUC of 0.986. The AP for breast lesion detection was 84.1% using weakly supervised learning. Weakly supervised learning based on FPN combined with Layer-CAM facilitated automatic detection of breast lesion.

Breast Cancer Surrogate Subtype Classification Using Pretreatment Multi-Phase Dynamic Contrast-Enhanced Magnetic Resonance Imaging Radiomics: A Retrospective Single-Center Study

This study aimed to explore the potential of multi-phase dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) radiomics for classifying breast cancer surrogate subtypes. This retrospective study analyzed 360 breast cancers from 319 patients who underwent pretreatment DCE-MRI between January 2015 and January 2019. The cohort consisted of 33 triple-negative, 26 human epidermal growth factor receptor 2 (HER2)-positive, 109 luminal A-like, 144 luminal B-like HER2-negative, and 48 luminal B-like HER2-positive lesions. A total of 1781 radiomic features were extracted from manually segmented breast cancers in each DCE-MRI sequence. The model was internally validated and selected using ten times repeated five-fold cross-validation on the primary cohort, with further evaluation using a validation cohort. The most successful models were logistic regression models applied to the third post-contrast subtraction images. These models exhibited the highest area under the curve (AUC) for discriminating between luminal A like vs. others (AUC: 0.78), luminal B-like HER2 negative vs. others (AUC: 0.57), luminal B-like HER2 positive vs. others (AUC: 0.60), HER2 positive vs. others (AUC: 0.81), and triple negative vs. others (AUC: 0.83). In conclusion, the radiomic features extracted from multi-phase DCE-MRI are promising for discriminating between breast cancer subtypes. The best-performing models relied on tissue changes observed during the mid-stage of the imaging process.

Assessing the Role of Different Heterogeneous Regions in DCE-MRI for Predicting Molecular Subtypes of Breast Cancer based on Network Architecture Search and Vision Transformer

Breast cancer, the most common female malignancy, is highly heterogeneous, manifesting as different molecular subtypes. It is clinically important to distinguish between these molecular subtypes due to marked differences in prognosis, treatment and survival outcomes. In this study, we first performed convex analysis of mixtures (CAM) analysis on both intratumoral and peritumoral regions in DCE-MRI to generate multiple heterogeneous regions. Then, we developed a vision transformer (ViT)-based DL model and performed network architecture search (NAS) to evaluate all the combination of different heterogeneous regions for predicting molecular subtypes of breast cancer. Experimental results showed that the input plasma from both peritumoral and intratumoral regions, and the fast-flow kinetics from intratumoral regions were critical for predicting different molecular subtypes, achieving an area under receiver operating characteristic curve (AUROC) value of 0.66-0.68.Clinical Relevance- This study reduces the redundancy in multiple heterogeneous subregions and supports the precise prediction of molecular subtypes, which is of potential importance for the medicine care and treatment planning of patients with breast cancer.

Clinical applications of deep learning in breast MRI

Deep learning (DL) is one of the most powerful data-driven machine-learning techniques in artificial intelligence (AI). It can automatically learn from raw data without manual feature selection. DL models have led to remarkable advances in data extraction and analysis for medical imaging. Magnetic resonance imaging (MRI) has proven useful in delineating the characteristics and extent of breast lesions and tumors. This review summarizes the current state-of-the-art applications of DL models in breast MRI. Many recent DL models were examined in this field, along with several advanced learning approaches and methods for data normalization and breast and lesion segmentation. For clinical applications, DL-based breast MRI models were proven useful in five aspects: diagnosis of breast cancer, classification of molecular types, classification of histopathological types, prediction of neoadjuvant chemotherapy response, and prediction of lymph node metastasis. For subsequent studies, further improvement in data acquisition and preprocessing is necessary, additional DL techniques in breast MRI should be investigated, and wider clinical applications need to be explored.

Analysis of Breast Cancer Differences between China and Western Countries Based on Radiogenomics

Using radiogenomics methods, the differences between tumor imaging data and genetic data in Chinese and Western breast cancer (BC) patients were analyzed, and the correlation between phenotypic data and genetic data was explored. In this paper, we analyzed BC patients' image characteristics and transcriptome data separately, then correlated the magnetic resonance imaging (MRI) phenotype with the transcriptome data through a computational method to develop a radiogenomics feature. The data was fed into the designed random forest (RF) model, which used the area under the receiver operating curve (AUC) as the evaluation index. Next, we analyzed the hub genes in the differentially expressed genes (DEGs) and obtained seven hub genes, which may cause Chinese and Western BC patients to behave differently in the clinic. We demonstrated that combining relevant genetic data and imaging features could better classify Chinese and Western patients than using genes or imaging characteristics alone. The AUC values of 0.74, 0.81, and 0.95 were obtained separately using the image characteristics, DEGs, and radiogenomics features. We screened SYT4, GABRG2, CHGA, SLC6A17, NEUROG2, COL2A1, and MATN4 and found that these genes were positively or negatively correlated with certain imaging characteristics. In addition, we found that the SLC6A17, NEUROG2, CHGA, and MATN4 genes were associated with clinical features.

Predicting hormone receptors and PAM50 subtypes of breast cancer from multi-scale lesion images of DCE-MRI with transfer learning technique

BACKGROUND

The recent development of artificial intelligence (AI) technologies coupled with medical imaging data has gained considerable attention, and offers a non-invasive approach for cancer diagnosis and prognosis. In this context, improved breast cancer (BC) molecular characteristics assessment models are foreseen to enable personalized strategies with better clinical outcomes compared to existing screening strategies. And it is a promising approach to developing models for hormone receptors (HR) and subtypes of BC patients from dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) data.

METHODS

In this institutional review board-approved study, 174 BC patients with both DCE-MRI and RNA-seq data in the local database were analyzed. Slice images from tumor lesions and multi-scale peri-tumor regions were used as model inputs, and five representative pre-trained transfer learning (TF) networks, such as Inception-v3 and Xception, were employed to establish prediction models. A comprehensive analysis was performed using five-fold cross-validation to avoid overfitting, and accuracy (ACC) and area under the receiver operating characteristic curve (AUROC) to evaluate model performance.

RESULTS

Xception achieved the superior results when using solely tumor regions, with highest AUROCs of 0.844 (95% CI: [0.841, 0.847]) and 0.784 (95% CI: [0.781, 0.788]) for estrogen receptor (ER) and progesterone receptor (PR), respectively, and best ACC of 0.467 (95% CI: [0.462, 0.470]) for PAM50 subtypes. A significant improvement in the model performance was observed when images of the peri-tumor region were included, with optimal results achieved using images of the tumor and the 10 mm peri-tumor regions. Xception-based TF models performed most effectively in predicting ER and PR statuses, with the AUROCs were 0.942 (95% CI: [0.940, 0.944]) and 0.920 (95% CI: [0.917, 0.922]), respectively, whereas for PAM50 subtypes, the Inception-v3-based network yielded the highest ACC as 0.742 (95% CI: [0.738, 0.746]).

CONCLUSIONS

Transfer learning analysis based on DCE-MRI data of tumor and peri-tumor regions was helpful to the non-invasive assessment of molecular characteristics of BC.

Noninvasive assessment of breast cancer molecular subtypes on multiparametric MRI using convolutional neural network with transfer learning

BACKGROUND

To evaluate the performances of multiparametric MRI-based convolutional neural networks (CNNs) for the preoperative assessment of breast cancer molecular subtypes.

METHODS

A total of 136 patients with 136 pathologically confirmed invasive breast cancers were randomly divided into training, validation, and testing sets in this retrospective study. The CNN models were established based on contrast-enhanced T₁ -weighted imaging (T₁ C), Apparent diffusion coefficient (ADC), and T₂ -weighted imaging (T₂ W) using the training and validation sets. The performances of CNN models were evaluated on the testing set. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and accuracy were calculated to assess the performance.

RESULTS

For the separation of each subtype from other subtypes on the testing set, the T₁ C-based models yielded AUCs from 0.762 to 0.920; the ADC-based models yielded AUCs from 0.686 to 0.851; and the T₂ W-based models achieved AUCs from 0.639 to 0.697.

CONCLUSION

T₁ C-based models performed better than ADC-based models and T₂ W-based models in assessing the breast cancer molecular subtypes. The discriminating performances of our CNN models for triple negative and human epidermal growth factor receptor 2-enriched subtypes were better than that of luminal A and luminal B subtypes.

The Role of Deep Learning in Advancing Breast Cancer Detection Using Different Imaging Modalities: A Systematic Review

Breast cancer is among the most common and fatal diseases for women, and no permanent treatment has been discovered. Thus, early detection is a crucial step to control and cure breast cancer that can save the lives of millions of women. For example, in 2020, more than 65% of breast cancer patients were diagnosed in an early stage of cancer, from which all survived. Although early detection is the most effective approach for cancer treatment, breast cancer screening conducted by radiologists is very expensive and time-consuming. More importantly, conventional methods of analyzing breast cancer images suffer from high false-detection rates. Different breast cancer imaging modalities are used to extract and analyze the key features affecting the diagnosis and treatment of breast cancer. These imaging modalities can be divided into subgroups such as mammograms, ultrasound, magnetic resonance imaging, histopathological images, or any combination of them. Radiologists or pathologists analyze images produced by these methods manually, which leads to an increase in the risk of wrong decisions for cancer detection. Thus, the utilization of new automatic methods to analyze all kinds of breast screening images to assist radiologists to interpret images is required. Recently, artificial intelligence (AI) has been widely utilized to automatically improve the early detection and treatment of different types of cancer, specifically breast cancer, thereby enhancing the survival chance of patients. Advances in AI algorithms, such as deep learning, and the availability of datasets obtained from various imaging modalities have opened an opportunity to surpass the limitations of current breast cancer analysis methods. In this article, we first review breast cancer imaging modalities, and their strengths and limitations. Then, we explore and summarize the most recent studies that employed AI in breast cancer detection using various breast imaging modalities. In addition, we report available datasets on the breast-cancer imaging modalities which are important in developing AI-based algorithms and training deep learning models. In conclusion, this review paper tries to provide a comprehensive resource to help researchers working in breast cancer imaging analysis.

Brain Tumor Characterization Using Radiogenomics in Artificial Intelligence Framework

Brain tumor characterization (BTC) is the process of knowing the underlying cause of brain tumors and their characteristics through various approaches such as tumor segmentation, classification, detection, and risk analysis. The substantial brain tumor characterization includes the identification of the molecular signature of various useful genomes whose alteration causes the brain tumor. The radiomics approach uses the radiological image for disease characterization by extracting quantitative radiomics features in the artificial intelligence (AI) environment. However, when considering a higher level of disease characteristics such as genetic information and mutation status, the combined study of "radiomics and genomics" has been considered under the umbrella of "radiogenomics". Furthermore, AI in a radiogenomics' environment offers benefits/advantages such as the finalized outcome of personalized treatment and individualized medicine. The proposed study summarizes the brain tumor's characterization in the prospect of an emerging field of research, i.e., radiomics and radiogenomics in an AI environment, with the help of statistical observation and risk-of-bias (RoB) analysis. The PRISMA search approach was used to find 121 relevant studies for the proposed review using IEEE, Google Scholar, PubMed, MDPI, and Scopus. Our findings indicate that both radiomics and radiogenomics have been successfully applied aggressively to several oncology applications with numerous advantages. Furthermore, under the AI paradigm, both the conventional and deep radiomics features have made an impact on the favorable outcomes of the radiogenomics approach of BTC. Furthermore, risk-of-bias (RoB) analysis offers a better understanding of the architectures with stronger benefits of AI by providing the bias involved in them.

Development and validation of a deep learning model for breast lesion segmentation and characterization in multiparametric MRI

Importance

The utilization of artificial intelligence for the differentiation of benign and malignant breast lesions in multiparametric MRI (mpMRI) assists radiologists to improve diagnostic performance.

Objectives

To develop an automated deep learning model for breast lesion segmentation and characterization and to evaluate the characterization performance of AI models and radiologists.

Materials and methods

For lesion segmentation, 2,823 patients were used for the training, validation, and testing of the VNet-based segmentation models, and the average Dice similarity coefficient (DSC) between the manual segmentation by radiologists and the mask generated by VNet was calculated. For lesion characterization, 3,303 female patients with 3,607 pathologically confirmed lesions (2,213 malignant and 1,394 benign lesions) were used for the three ResNet-based characterization models (two single-input and one multi-input models). Histopathology was used as the diagnostic criterion standard to assess the characterization performance of the AI models and the BI-RADS categorized by the radiologists, in terms of sensitivity, specificity, accuracy, and the area under the receiver operating characteristic curve (AUC). An additional 123 patients with 136 lesions (81 malignant and 55 benign lesions) from another institution were available for external testing.

Results

Of the 5,811 patients included in the study, the mean age was 46.14 (range 11–89) years. In the segmentation task, a DSC of 0.860 was obtained between the VNet-generated mask and manual segmentation by radiologists. In the characterization task, the AUCs of the multi-input and the other two single-input models were 0.927, 0.821, and 0.795, respectively. Compared to the single-input DWI or DCE model, the multi-input DCE and DWI model obtained a significant increase in sensitivity, specificity, and accuracy (0.831 vs. 0.772/0.776, 0.874 vs. 0.630/0.709, 0.846 vs. 0.721/0.752). Furthermore, the specificity of the multi-input model was higher than that of the radiologists, whether using BI-RADS category 3 or 4 as a cutoff point (0.874 vs. 0.404/0.841), and the accuracy was intermediate between the two assessment methods (0.846 vs. 0.773/0.882). For the external testing, the performance of the three models remained robust with AUCs of 0.812, 0.831, and 0.885, respectively.

Conclusions

Combining DCE with DWI was superior to applying a single sequence for breast lesion characterization. The deep learning computer-aided diagnosis (CADx) model we developed significantly improved specificity and achieved comparable accuracy to the radiologists with promise for clinical application to provide preliminary diagnoses.

One-step algorithm for fast-track localization and multi-category classification of histological subtypes in lung cancer

BACKGROUNDS

Accumulated evidence has proven that computer-derived features from computed tomography (CT) through radiomics and deep learning technologies can identify extensive characteristics of pulmonary malignancies, such as nodules detection and malignant lesion discrimination. However, there are few studies on whether CT images can reflect histological subtypes of lung cancer through computer-derived features.

METHODS

Contrast-enhanced CT images prior treatment from 417 patients diagnosed with small cell lung cancer (SCLC), lung adenocarcinoma (ADC), or lung squamous cell carcinoma (SCC) were collected. ITK-SNAP software was used by trained radiologists for the manual delineation of tumor volume. Patients of each category (SCLC, ADC, SCC) were then randomly split into training datasets and test datasets in an approximately ratio of 8:2. After image pre-processing and augmentation, 25,042 CT images from the training datasets were used to train our self-developed deep learning model for fast-tracking tumor lesions and classifying corresponding histological subtypes simultaneously. The performance of the network was evaluated by accuracy, F1-score and weighted F1-average using 1,921 testing images based on parameters generated during training.

RESULTS

The prediction accuracy of SCLC, ADC, and SCC were 0.83, 0.75 and 0.67, respectively. The weighted F1-average was 0.75. ADC obtained the best F1-score of 0.78, which was outperformed SCLC (0.77) and SCC (0.66). The corresponding AUC values of SCLC, ADC, and SCC were 0.87, 0.84, and 0.76, respectively. Only 0.24 s were required to simultaneously achieve functions of tumor localization and histological classification on a thoracic CT image slice. The heat map visualization illustrated the extracted tumor features to classify subtypes of lung cancer by the proposed model.

CONCLUSIONS

The newly developed multi-task algorithm provides a CNN-based DL approach in lung cancer for automatically fast-tracking tumor lesions and classifying corresponding histological subtypes in one-step.

Role of Artificial Intelligence in Radiogenomics for Cancers in the Era of Precision Medicine

Simple Summary

Recently, radiogenomics has played a significant role and offered a new understanding of cancer’s biology and behavior in response to standard therapy. It also provides a more precise prognosis, investigation, and analysis of the patient’s cancer. Over the years, Artificial Intelligence (AI) has provided a significant strength in radiogenomics. In this paper, we offer computational and oncological prospects of the role of AI in radiogenomics, as well as its offers, achievements, opportunities, and limitations in the current clinical practices.

Abstract

Radiogenomics, a combination of “Radiomics” and “Genomics,” using Artificial Intelligence (AI) has recently emerged as the state-of-the-art science in precision medicine, especially in oncology care. Radiogenomics syndicates large-scale quantifiable data extracted from radiological medical images enveloped with personalized genomic phenotypes. It fabricates a prediction model through various AI methods to stratify the risk of patients, monitor therapeutic approaches, and assess clinical outcomes. It has recently shown tremendous achievements in prognosis, treatment planning, survival prediction, heterogeneity analysis, reoccurrence, and progression-free survival for human cancer study. Although AI has shown immense performance in oncology care in various clinical aspects, it has several challenges and limitations. The proposed review provides an overview of radiogenomics with the viewpoints on the role of AI in terms of its promises for computational as well as oncological aspects and offers achievements and opportunities in the era of precision medicine. The review also presents various recommendations to diminish these obstacles.

Breast Tumor Detection Using Robust and Efficient Machine Learning and Convolutional Neural Network Approaches

Breast cancer develops when cells in the breast expand and divide uncontrollably, resulting in a lump of tissue known as a tumor. This lump of tissue is called a tumor. After skin cancer, breast cancer is the second most common cancer among women. It is more common in women over the age of 50. Men may also acquire breast cancer, albeit it is uncommon. Each year, approximately 2,600 men in the United States are diagnosed with breast cancer, accounting for less than 1% of all cases. Transgender women are more likely than cisgender men to acquire breast cancer. Additionally, transgender males are less likely than cisgender women to acquire breast cancer. Breast cancer is more common in women over the age of 50, although it can affect anyone at any age. Early detection of a breast tumor may significantly lower the risk of developing breast cancer. A public dataset of breast tumor features was used instead to build models for identifying breast tumors through machine learning and deep learning. Prediction models were built using logistic regression (LR), decision tree (DT), random forest (RF), voting classifier (VC), support vector machine (SVM), and a proprietary convolutional neural network (CNN). These models were used to find critical prognostic indicators linked to breast cancer. The proposed network performs far better, with an average accuracy of 99%. This study has six types of models: LR, RF, SVM, VC, DT, and a custom CNN model. They all had 96% to 99% accuracy in this study. CNN, LR, RF, SVM, VC, and DT achieved 99%, 96%, 98%, 97%, 97%, and 96% F1 score, respectively. There were many machine learning algorithms used in this study that were very accurate, which means that these techniques could be used as alternative prognostic tools in breast tumor detection studies in Asia.

Structural and diffusion MRI based schizophrenia classification using 2D pretrained and 3D naive Convolutional Neural Networks

The ability of automatic feature learning makes Convolutional Neural Network (CNN) potentially suitable to uncover the complex and widespread brain changes in schizophrenia. Despite that, limited studies have been done on schizophrenia identification using interpretable deep learning approaches on multimodal neuroimaging data. Here, we developed a deep feature approach based on pre-trained 2D CNN and naive 3D CNN models trained from scratch for schizophrenia classification by integrating 3D structural and diffusion magnetic resonance imaging (MRI) data. We found that the naive 3D CNN models outperformed the pretrained 2D CNN models and the handcrafted feature-based machine learning approach using support vector machine during both cross-validation and testing on an independent dataset. Multimodal neuroimaging-based models accomplished performance superior to models based on a single modality. Furthermore, we identified brain grey matter and white matter regions critical for illness classification at the individual- and group-level which supported the salience network and striatal dysfunction hypotheses in schizophrenia. Our findings underscore the potential of CNN not only to automatically uncover and integrate multimodal 3D brain imaging features for schizophrenia identification, but also to provide relevant neurobiological interpretations which are crucial for developing objective and interpretable imaging-based probes for prognosis and diagnosis in psychiatric disorders.

Integration of Omics and Phenotypic Data for Precision Medicine

Over the past two decades, biomedical research is moving toward a big-data-driven approach. The underlying causes of this transition include the ability to gather genetic or molecular profiles of humans faster, the increasing adoption of electronic health record (EHR) system, and the growing interest in linking omics and phenotypic data for analysis. The integration of individual's biology data (e.g., genomics, proteomics, metabolomics), and health-care data has created unprecedented opportunities for precision medicine, that is, a medical model that uses a patient's unique information, mainly genetic, to prevent, diagnose, or treat disease. This chapter reviewed the research opportunities and applications of integrating omics and phenotypic data for precision medicine, such as understanding the relationship between genotype and phenotype, disease subtyping, and diagnosis or prediction of adverse outcomes. We reviewed the recent advanced methods, particularly the machine learning and deep learning-based approaches used for harnessing and harmonizing the multiomics and phenotypic data to address these applications. We finally discussed the challenges and future directions.

Application of Deep Learning in Breast Cancer Imaging

This review gives an overview of the current state of deep learning research in breast cancer imaging. Breast imaging plays a major role in detecting breast cancer at an earlier stage, as well as monitoring and evaluating breast cancer during treatment. The most commonly used modalities for breast imaging are digital mammography, digital breast tomosynthesis, ultrasound and magnetic resonance imaging. Nuclear medicine imaging techniques are used for detection and classification of axillary lymph nodes and distant staging in breast cancer imaging. All of these techniques are currently digitized, enabling the possibility to implement deep learning (DL), a subset of Artificial intelligence, in breast imaging. DL is nowadays embedded in a plethora of different tasks, such as lesion classification and segmentation, image reconstruction and generation, cancer risk prediction, and prediction and assessment of therapy response. Studies show similar and even better performances of DL algorithms compared to radiologists, although it is clear that large trials are needed, especially for ultrasound and magnetic resonance imaging, to exactly determine the added value of DL in breast cancer imaging. Studies on DL in nuclear medicine techniques are only sparsely available and further research is mandatory. Legal and ethical issues need to be considered before the role of DL can expand to its full potential in clinical breast care practice.

Fused feature signatures to probe tumour radiogenomics relationships

Radiogenomics relationships (RRs) aims to identify statistically significant correlations between medical image features and molecular characteristics from analysing tissue samples. Previous radiogenomics studies mainly relied on a single category of image feature extraction techniques (ETs); these are (i) handcrafted ETs that encompass visual imaging characteristics, curated from knowledge of human experts and, (ii) deep ETs that quantify abstract-level imaging characteristics from large data. Prior studies therefore failed to leverage the complementary information that are accessible from fusing the ETs. In this study, we propose a fused feature signature (FFSig): a selection of image features from handcrafted and deep ETs (e.g., transfer learning and fine-tuning of deep learning models). We evaluated the FFSig's ability to better represent RRs compared to individual ET approaches with two public datasets: the first dataset was used to build the FFSig using 89 patients with non-small cell lung cancer (NSCLC) comprising of gene expression data and CT images of the thorax and the upper abdomen for each patient; the second NSCLC dataset comprising of 117 patients with CT images and RNA-Seq data and was used as the validation set. Our results show that our FFSig encoded complementary imaging characteristics of tumours and identified more RRs with a broader range of genes that are related to important biological functions such as tumourigenesis. We suggest that the FFSig has the potential to identify important RRs that may assist cancer diagnosis and treatment in the future.

Extendable and explainable deep learning for pan-cancer radiogenomics research

Radiogenomics is a field where medical images and genomic profiles are jointly analyzed to answer critical clinical questions. Specifically, people want to identify non-invasive imaging biomarkers that are associated with both genomic features and clinical outcomes. Deep learning is an advanced computer science technique that has been applied in many fields, including medical image and genomic data analysis. This review summarizes the current state of deep learning in pan-cancer radiogenomic research, discusses its limitations, and indicates the potential future directions. Traditional machine learning in radiomics, genomics, and radiogenomics have also been briefly discussed. We also summarize the main pan-cancer radiogenomic research resources. Two characteristics of deep learning are emphasized when discussing its application to pan-cancer radiogenomics, which are extendibility and explainability.

A computer-aided diagnosis system for the classification of COVID-19 and non-COVID-19 pneumonia on chest X-ray images by integrating CNN with sparse autoencoder and feed forward neural network

Several infectious diseases have affected the lives of many people and have caused great dilemmas all over the world. COVID-19 was declared a pandemic caused by a newly discovered virus named Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) by the World Health Organisation in 2019. RT-PCR is considered the golden standard for COVID-19 detection. Due to the limited RT-PCR resources, early diagnosis of the disease has become a challenge. Radiographic images such as Ultrasound, CT scans, X-rays can be used for the detection of the deathly disease. Developing deep learning models using radiographic images for detecting COVID-19 can assist in countering the outbreak of the virus. This paper presents a computer-aided detection model utilizing chest X-ray images for combating the pandemic. Several pre-trained networks and their combinations have been used for developing the model. The method uses features extracted from pre-trained networks along with Sparse autoencoder for dimensionality reduction and a Feed Forward Neural Network (FFNN) for the detection of COVID-19. Two publicly available chest X-ray image datasets, consisting of 504 COVID-19 images and 542 non-COVID-19 images, have been combined to train the model. The method was able to achieve an accuracy of 0.9578 and an AUC of 0.9821, using the combination of InceptionResnetV2 and Xception. Experiments have proved that the accuracy of the model improves with the usage of sparse autoencoder as the dimensionality reduction technique.

Training Strategies for Radiology Deep Learning Models in Data-limited Scenarios

Data-driven approaches have great potential to shape future practices in radiology. The most straightforward strategy to obtain clinically accurate models is to use large, well-curated and annotated datasets. However, patient privacy constraints, tedious annotation processes, and the limited availability of radiologists pose challenges to building such datasets. This review details model training strategies in scenarios with limited data, insufficiently labeled data, and/or limited expert resources. This review discusses strategies to enlarge the data sample, decrease the time burden of manual supervised labeling, adjust the neural network architecture to improve model performance, apply semisupervised approaches, and leverage efficiencies from pretrained models. Keywords: Computer-aided Detection/Diagnosis, Transfer Learning, Limited Annotated Data, Augmentation, Synthetic Data, Semisupervised Learning, Federated Learning, Few-Shot Learning, Class Imbalance.

Radiomic differentiation of breast cancer molecular subtypes using pre-operative breast imaging - A systematic review and meta-analysis

INTRODUCTION

Breast cancer has four distinct molecular subtypes which are discriminated using gene expression profiling following biopsy. Radiogenomics is an emerging field which utilises diagnostic imaging to reveal genomic properties of disease. We aimed to perform a systematic review of the current literature to evaluate the value radiomics in differentiating breast cancers into their molecular subtypes using diagnostic imaging.

METHODS

A systematic review was performed as per PRISMA guidelines. Studies assessing radiomictumour analysis in differentiatingbreast cancer molecular subtypeswere included. Quality was assessed using the radiomics quality score (RQS). Diagnostic sensitivity and specificity of radiomic analyses were included for meta-analysis; Study specific sensitivity and specificity were retrieved and summary ROC analysis were performed to compile pooled sensitivities and specificities.

RESULTS

Forty-one studies were included. Overall, there were 10,090 female patients (mean age of 47.6 ± 11.7 years, range: 21-93) and molecular subtypewas reported in 7,693 of cases, with Luminal A (LABC), Luminal B (LBBC), Human Epidermal Growth Factor Receptor-2 overexpressing (HER2+), and Triple Negative (TNBC) breast cancers representing 51.3%, 19.9%, 12.3% and 16.3% of tumour respectively. Seven studies provided radiomic analysis to determine molecular subtypes using mammography to differentiateTNBCvs.others (sensitivity: 0.82,specificity:0.79). Thirty-five studies reported on radiomic analysis of magnetic resonance imaging (MRI); LABC versus others(sensitivity:0.78,specificity:0.83),HER2+versusothers(sensitivity:0.87,specificity:0.88), andLBBCversusTNBC (sensitivity: 0.79,specificity:0.88) respectively.

CONCLUSION

Radiomic tumour assessment of contemporary breast imaging provide a novel option in determining breast cancer molecular subtypes. However, amelioration of such techniques are required and genetic expression assessment will remain the gold standard.

Prostate Cancer Radiogenomics-From Imaging to Molecular Characterization

Radiomics and genomics represent two of the most promising fields of cancer research, designed to improve the risk stratification and disease management of patients with prostate cancer (PCa). Radiomics involves a conversion of imaging derivate quantitative features using manual or automated algorithms, enhancing existing data through mathematical analysis. This could increase the clinical value in PCa management. To extract features from imaging methods such as magnetic resonance imaging (MRI), the empiric nature of the analysis using machine learning and artificial intelligence could help make the best clinical decisions. Genomics information can be explained or decoded by radiomics. The development of methodologies can create more-efficient predictive models and can better characterize the molecular features of PCa. Additionally, the identification of new imaging biomarkers can overcome the known heterogeneity of PCa, by non-invasive radiological assessment of the whole specific organ. In the future, the validation of recent findings, in large, randomized cohorts of PCa patients, can establish the role of radiogenomics. Briefly, we aimed to review the current literature of highly quantitative and qualitative results from well-designed studies for the diagnoses, treatment, and follow-up of prostate cancer, based on radiomics, genomics and radiogenomics research.

AI-Enhanced Diagnosis of Challenging Lesions in Breast MRI: A Methodology and Application Primer

Computer-aided diagnosis (CAD) systems have become an important tool in the assessment of breast tumors with magnetic resonance imaging (MRI). CAD systems can be used for the detection and diagnosis of breast tumors as a "second opinion" review complementing the radiologist's review. CAD systems have many common parts, such as image preprocessing, tumor feature extraction, and data classification that are mostly based on machine-learning (ML) techniques. In this review article, we describe applications of ML-based CAD systems in MRI covering the detection of diagnostically challenging lesions of the breast such as nonmass enhancing (NME) lesions, and furthermore discuss how multiparametric MRI and radiomics can be applied to the study of NME, including prediction of response to neoadjuvant chemotherapy (NAC). Since ML has been widely used in the medical imaging community, we provide an overview about the state-of-the-art and novel techniques applied as classifiers to CAD systems. The differences in the CAD systems in MRI of the breast for several standard and novel applications for NME are explained in detail to provide important examples, illustrating: 1) CAD for detection and diagnosis, 2) CAD in multiparametric imaging, 3) CAD in NAC, and 4) breast cancer radiomics. We aim to provide a comparison between these CAD applications and to illustrate a global view on intelligent CAD systems based on machine and deep learning in MRI of the breast. LEVEL OF EVIDENCE: 2 TECHNICAL EFFICACY STAGE: 2.

Prediction of breast cancer molecular subtypes using DCE-MRI based on CNNs combined with ensemble learning

To design an ensemble learning based prediction model using different breast DCE-MR post-contrast sequence images to distinguish two kinds of breast cancer subtypes (luminal and non-luminal). We retrospectively studied preoperative dynamic contrast enhanced-magnetic resonance imaging and molecular information of 266 breast cancer cases with either luminal subtype (luminal A and luminal B) or non-luminal subtype (human epidermal growth factor receptor 2 and triple negative). Then, multiple bounding boxes covering tumor lesions were acquired from three series of post-contrast DCE-MR sequence images which were determined by radiologists. Afterwards, three baseline convolutional neural networks (CNNs) with same architecture were concurrently trained, followed by preliminary prediction of probabilities from the testing database. Finally, the classification and evaluation of breast subtypes were realized by means of fusing predicted results from three CNNs employed via ensemble learning based on weighted voting. Taking advantage of 5-fold cross validation CV, the average prediction specificity, accuracy, precision and area under the ROC curve on testing dataset for the luminal versus non-luminal are 0.958, 0.852, 0.961, and 0.867, respectively, which empirically demonstrate that our proposed ensemble model has highly reliability and robustness. The breast DCE-MR post-contrast sequence image analysis utilizing the ensemble CNN model based on deep learning could show a valuable and extendible practical application on breast molecular subtype identification.

Personalized brachytherapy dose reconstruction using deep learning

BACKGROUND AND PURPOSE

Accurate calculation of the absorbed dose delivered to the tumor and normal tissues improves treatment gain factor, which is the major advantage of brachytherapy over external radiation therapy. To address the simplifications of TG-43 assumptions that ignore the dosimetric impact of medium heterogeneities, we proposed a deep learning (DL)-based approach, which improves the accuracy while requiring a reasonable computation time.

MATERIALS AND METHODS

We developed a Monte Carlo (MC)-based personalized brachytherapy dosimetry simulator (PBrDoseSim), deployed to generate patient-specific dose distributions. A deep neural network (DNN) was trained to predict personalized dose distributions derived from MC simulations, serving as ground truth. The paired channel input used for the training is composed of dose distribution kernel in water medium along with the full-volumetric density maps obtained from CT images reflecting medium heterogeneity.

RESULTS

The predicted single-dwell dose kernels were in good agreement with MC-based kernels serving as reference, achieving a mean relative absolute error (MRAE) and mean absolute error (MAE) of 1.16 ± 0.42% and 4.2 ± 2.7 × 10^-4 (Gy.sec^-1/voxel), respectively. The MRAE of the dose volume histograms (DVHs) between the DNN and MC calculations in the clinical target volume were 1.8 ± 0.86%, 0.56 ± 0.56%, and 1.48 ± 0.72% for D90, V150, and V100, respectively. For bladder, sigmoid, and rectum, the MRAE of D5cc between the DNN and MC calculations were 2.7 ± 1.7%, 1.9 ± 1.3%, and 2.1 ± 1.7%, respectively.

CONCLUSION

The proposed DNN-based personalized brachytherapy dosimetry approach exhibited comparable performance to the MC method while overcoming the computational burden of MC calculations and oversimplifications of TG-43.

Topic Evolution Analysis for Omics Data Integration in Cancers

One of the vital challenges for cancer diseases is efficient biomarkers monitoring formation and development are limited. Omics data integration plays a crucial role in the mining of biomarkers in the human condition. As the link between omics study on biomarkers discovery and cancer diseases is deepened, defining the principal technologies applied in the field is a must not only for the current period but also for the future. We utilize topic modeling to extract topics (or themes) as a probabilistic distribution of latent topics from the dataset. To predict the future trend of related cases, we utilize the Prophet neural network to perform a prediction correction model for existing topics. A total of 2,318 pieces of literature (from 2006 to 2020) were retrieved from MEDLINE with the query on “omics” and “cancer.” Our study found 20 topics covering current research types. The topic extraction results indicate that, with the rapid development of omics data integration research, multi-omics analysis (Topic 11) and genomics of colorectal cancer (Topic 10) have more studies reported last 15 years. From the topic prediction view, research findings in multi-omics data processing and novel biomarker discovery for cancer prediction (Topic 2, 3, 10, 11) will be heavily focused in the future. From the topic visuallization and evolution trends, metabolomics of breast cancer (Topic 9), pharmacogenomics (Topic 15), genome-guided therapy regimens (Topic 16), and microRNAs target genes (Topic 17) could have more rapidly developed in the study of cancer treatment effect and recurrence prediction.

Evaluating the Accuracy of Breast Cancer and Molecular Subtype Diagnosis by Ultrasound Image Deep Learning Model

Background: Breast ultrasound is the first choice for breast tumor diagnosis in China, but the Breast Imaging Reporting and Data System (BI-RADS) categorization routinely used in the clinic often leads to unnecessary biopsy. Radiologists have no ability to predict molecular subtypes with important pathological information that can guide clinical treatment.

Materials and Methods: This retrospective study collected breast ultrasound images from two hospitals and formed training, test and external test sets after strict selection, which included 2,822, 707, and 210 ultrasound images, respectively. An optimized deep learning model (DLM) was constructed with the training set, and the performance was verified in both the test set and the external test set. Diagnostic results were compared with the BI-RADS categorization determined by radiologists. We divided breast cancer into different molecular subtypes according to hormone receptor (HR) and human epidermal growth factor receptor 2 (HER2) expression. The ability to predict molecular subtypes using the DLM was confirmed in the test set.

Results: In the test set, with pathological results as the gold standard, the accuracy, sensitivity and specificity were 85.6, 98.7, and 63.1%, respectively, according to the BI-RADS categorization. The same set achieved an accuracy, sensitivity, and specificity of 89.7, 91.3, and 86.9%, respectively, when using the DLM. For the test set, the area under the curve (AUC) was 0.96. For the external test set, the AUC was 0.90. The diagnostic accuracy was 92.86% with the DLM in BI-RADS 4a patients. Approximately 70.76% of the cases were judged as benign tumors. Unnecessary biopsy was theoretically reduced by 67.86%. However, the false negative rate was 10.4%. A good prediction effect was shown for the molecular subtypes of breast cancer with the DLM. The AUC were 0.864, 0.811, and 0.837 for the triple-negative subtype, HER2 (+) subtype and HR (+) subtype predictions, respectively.

Conclusion: This study showed that the DLM was highly accurate in recognizing breast tumors from ultrasound images. Thus, the DLM can greatly reduce the incidence of unnecessary biopsy, especially for patients with BI-RADS 4a. In addition, the predictive ability of this model for molecular subtypes was satisfactory,which has specific clinical application value.

Boosted EfficientNet: Detection of Lymph Node Metastases in Breast Cancer Using Convolutional Neural Networks

(1) Purpose: To improve the capability of EfficientNet, including developing a cropping method called Random Center Cropping (RCC) to retain the original image resolution and significant features on the images' center area, reducing the downsampling scale of EfficientNet to facilitate the small resolution images of RPCam datasets, and integrating attention and Feature Fusion (FF) mechanisms with EfficientNet to obtain features containing rich semantic information. (2) Methods: We adopt the Convolutional Neural Network (CNN) to detect and classify lymph node metastasis in breast cancer. (3) Results: Experiments illustrate that our methods significantly boost performance of basic CNN architectures, where the best-performed method achieves an accuracy of 97.96% ± 0.03% and an Area Under the Curve (AUC) of 99.68% ± 0.01% on RPCam datasets, respectively. (4) Conclusions: (1) To our limited knowledge, we are the only study to explore the power of EfficientNet on Metastatic Breast Cancer (MBC) classification, and elaborate experiments are conducted to compare the performance of EfficientNet with other state-of-the-art CNN models. It might provide inspiration for researchers who are interested in image-based diagnosis using Deep Learning (DL). (2) We design a novel data augmentation method named RCC to promote the data enrichment of small resolution datasets. (3) All of our four technological improvements boost the performance of the original EfficientNet.

The Era of Radiogenomics in Precision Medicine: An Emerging Approach to Support Diagnosis, Treatment Decisions, and Prognostication in Oncology

With the rapid development of new technologies, including artificial intelligence and genome sequencing, radiogenomics has emerged as a state-of-the-art science in the field of individualized medicine. Radiogenomics combines a large volume of quantitative data extracted from medical images with individual genomic phenotypes and constructs a prediction model through deep learning to stratify patients, guide therapeutic strategies, and evaluate clinical outcomes. Recent studies of various types of tumors demonstrate the predictive value of radiogenomics. And some of the issues in the radiogenomic analysis and the solutions from prior works are presented. Although the workflow criteria and international agreed guidelines for statistical methods need to be confirmed, radiogenomics represents a repeatable and cost-effective approach for the detection of continuous changes and is a promising surrogate for invasive interventions. Therefore, radiogenomics could facilitate computer-aided diagnosis, treatment, and prediction of the prognosis in patients with tumors in the routine clinical setting. Here, we summarize the integrated process of radiogenomics and introduce the crucial strategies and statistical algorithms involved in current studies.

Diagnosis of cervical precancerous lesions based on multimodal feature changes

To realize the automatic diagnosis of cervical intraepithelial neoplasia (CIN) cases by preacetic acid test and postacetic acid test colposcopy images, this paper proposes a method of cervical precancerous lesion diagnosis based on multimodal feature changes. First, the preacetic acid test and postacetic acid test colposcopy images were registered based on cross-correlation and projection transformation, and then the cervical region was extracted by the k-means clustering algorithm. Finally, a deep learning network was used to extract features and classify the preacetic acid test and postacetic acid test cervical images after registration. Finally, the proposed method achieves a classification accuracy of 86.3%, a sensitivity of 84.1%, and a specificity of 89.8% in 60 test cases. Experimental results show that this method can make better use of the multimodal features of colposcopy images and has lower requirements for medical staff in the process of data acquisition. It has certain clinical significance in cervical cancer precancerous lesion screening systems.

Dyslexia detection using 3D convolutional neural networks and functional magnetic resonance imaging

BACKGROUND AND OBJECTIVES

Dyslexia is a disorder of neurological origin which affects the learning of those who suffer from it, mainly children, and causes difficulty in reading and writing. When undiagnosed, dyslexia leads to intimidation and frustration of the affected children and also of their family circles. In case no early intervention is given, children may reach high school with serious achievement gaps. Hence, early detection and intervention services for dyslexic students are highly important and recommended in order to support children in developing a positive self-esteem and reaching their maximum academic capacities. This paper presents a new approach for automatic recognition of children with dyslexia using functional magnetic resonance Imaging.

METHODS

Our proposed system is composed of a sequence of preprocessing steps to retrieve the brain activation areas during three different reading tasks. Conversion to Nifti volumes, adjustment of head motion, normalization and smoothing transformations were performed on the fMRI scans in order to bring all the subject brains into one single model which will enable voxels comparison between each subject. Subsequently, using Statistical Parametric Maps (SPMs), a total of 165 3D volumes containing brain activation of 55 children were created. The classification of these volumes was handled using three parallel 3D Convolutional Neural Network (3D CNN), each corresponding to a brain activation during one reading task, and concatenated in the last two dense layers, forming a single architecture devoted to performing optimized detection of dyslexic brain activation. Additionally, we used 4-fold cross validation method in order to assess the generalizability of our model and control overfitting.

RESULTS

Our approach has achieved an overall average classification accuracy of 72.73%, sensitivity of 75%, specificity of 71.43%, precision of 60% and an F1-score of 67% in dyslexia detection.

CONCLUSIONS

The proposed system has demonstrated that the recognition of dyslexic children is feasible using deep learning and functional magnetic resonance Imaging when performing phonological and orthographic reading tasks.