Axillary Lymph Node Evaluation Utilizing Convolutional Neural Networks Using MRI Dataset

Study on the differential diagnosis of benign and malignant breast lesions using a deep learning model based on multimodal images

OBJECTIVE

To establish a multimodal model for distinguishing benign and malignant breast lesions.

MATERIALS AND METHODS

Clinical data, mammography, and MRI images (including T2WI, diffusion-weighted images (DWI), apparent diffusion coefficient (ADC), and DCE-MRI images) of 132 benign and breast cancer patients were analyzed retrospectively. The region of interest (ROI) in each image was marked and segmented using MATLAB software. The mammography, T2WI, DWI, ADC, and DCE-MRI models based on the ResNet34 network were trained. Using an integrated learning method, the five models were used as a basic model, and voting methods were used to construct a multimodal model. The dataset was divided into a training set and a prediction set. The accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of the model were calculated. The diagnostic efficacy of each model was analyzed using a receiver operating characteristic curve (ROC) and an area under the curve (AUC). The diagnostic value was determined by the DeLong test with statistically significant differences set at P < 0.05.

RESULTS

We evaluated the ability of the model to classify benign and malignant tumors using the test set. The AUC values of the multimodal model, mammography model, T2WI model, DWI model, ADC model and DCE-MRI model were 0.943, 0.645, 0.595, 0.905, 0.900, and 0.865, respectively. The diagnostic ability of the multimodal model was significantly higher compared with that of the mammography and T2WI models. However, compared with the DWI, ADC, and DCE-MRI models, there was no significant difference in the diagnostic ability of these models.

CONCLUSION

Our deep learning model based on multimodal image training has practical value for the diagnosis of benign and malignant breast lesions.

Use of artificial intelligence in breast surgery: a narrative review

Background and Objective

We have witnessed tremendous advances in artificial intelligence (AI) technologies. Breast surgery, a subspecialty of general surgery, has notably benefited from AI technologies. This review aims to evaluate how AI has been integrated into breast surgery practices, to assess its effectiveness in improving surgical outcomes and operational efficiency, and to identify potential areas for future research and application.

Methods

Two authors independently conducted a comprehensive search of PubMed, Google Scholar, EMBASE, and Cochrane CENTRAL databases from January 1, 1950, to September 4, 2023, employing keywords pertinent to AI in conjunction with breast surgery or cancer. The search focused on English language publications, where relevance was determined through meticulous screening of titles, abstracts, and full-texts, followed by an additional review of references within these articles. The review covered a range of studies illustrating the applications of AI in breast surgery encompassing lesion diagnosis to postoperative follow-up. Publications focusing specifically on breast reconstruction were excluded.

Key Content and Findings

AI models have preoperative, intraoperative, and postoperative applications in the field of breast surgery. Using breast imaging scans and patient data, AI models have been designed to predict the risk of breast cancer and determine the need for breast cancer surgery. In addition, using breast imaging scans and histopathological slides, models were used for detecting, classifying, segmenting, grading, and staging breast tumors. Preoperative applications included patient education and the display of expected aesthetic outcomes. Models were also designed to provide intraoperative assistance for precise tumor resection and margin status assessment. As well, AI was used to predict postoperative complications, survival, and cancer recurrence.

Conclusions

Extra research is required to move AI models from the experimental stage to actual implementation in healthcare. With the rapid evolution of AI, further applications are expected in the coming years including direct performance of breast surgery. Breast surgeons should be updated with the advances in AI applications in breast surgery to provide the best care for their patients.

Application of artificial intelligence in ultrasound imaging for predicting lymph node metastasis in breast cancer: A meta-analysis

BACKGROUND

This study aims to comprehensively evaluate the accuracy and effectiveness of ultrasound imaging based on artificial intelligence algorithms in predicting lymph node metastasis in breast cancer patients through a meta-analysis.

METHODS

We systematically searched PubMed, Embase, and Cochrane Library for literature published up to May 2023. The search terms included artificial intelligence, ultrasound, breast cancer, and lymph node. Studies meeting the inclusion criteria were selected, and data were extracted for analysis. The main evaluation indicators included sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and area under the curve (AUC). The heterogeneity was assessed using the Cochrane Q test combined with the I^2 statistic expressing the percentage of total effect variation that can be attributed to the effect variation between studies, as recommended by the Cochrane Handbook for heterogeneity quantification. A threshold p-value of 0.10 was considered to compensate for the low power of the Q test. Sensitivity analysis was performed to assess the stability of individual studies, and publication bias was determined with funnel plots. Additionally, fagan plots were used to assess clinical utility.

RESULTS

Ten studies involving 4726 breast cancer patients were included in the meta-analysis. The results showed that ultrasound imaging based on artificial intelligence algorithms had high accuracy and effectiveness in predicting lymph node metastasis in breast cancer patients. The pooled sensitivity was 0.88 (95% CI: 0.81-0.93; P < 0.001; I2 = 84.68), specificity was 0.75 (95% CI: 0.66-0.83; P < 0.001; I2 = 91.11), and AUC was 0.89 (95% CI: 0.86-0.91). The positive likelihood ratio was 3.5 (95% CI: 2.6-4.8), negative likelihood ratio was 0.16 (95% CI: 0.10-0.26), and diagnostic odds ratio was 23 (95% CI: 13-40). However, the combined sensitivity of ultrasound imaging based on non-AI algorithms for predicting lymph node metastasis in breast cancer patients was 0.78 (95%CI: 0.63-0.88), the specificity was 0.76 (95%CI: 0.63-0.86), and the AUC was 0.84 (95%CI: 0.80-0.87). The positive likelihood ratio was 3.3 (95% CI: 1.9-5.6), the negative likelihood ratio was 0.29 (95% CI: 0.15-0.54), and the diagnostic odds ratio was 11 (95% CI: 4-33). Due to limited sample size (n = 2), meta-analysis was not conducted for the outcome of predicting lymph node metastasis burden.

CONCLUSION

Ultrasound imaging based on artificial intelligence algorithms holds promise in predicting lymph node metastasis in breast cancer patients, demonstrating high accuracy and effectiveness. The application of this technology helps in the diagnosis and treatment decisions for breast cancer patients and is expected to become an important tool in future clinical practice.

Development of a Diagnostic Artificial Intelligence Tool for Lateral Lymph Node Metastasis in Advanced Rectal Cancer

BACKGROUND

Metastatic lateral lymph node dissection can improve survival in patients with rectal adenocarcinoma, with or without chemoradiotherapy. However, the optimal imaging diagnostic criteria for lateral lymph node metastases remain undetermined.

OBJECTIVE

To develop a lateral lymph node metastasis diagnostic artificial intelligence tool using deep learning, for patients with rectal adenocarcinoma who underwent radical surgery and lateral lymph node dissection.

DESIGN

Retrospective study.

SETTINGS

Multicenter study.

PATIENTS

A total of 209 patients with rectal adenocarcinoma, who underwent radical surgery and lateral lymph node dissection at 15 participating hospitals, were enrolled in the study and allocated to training (n = 139), test (n = 17), or validation (n = 53) cohorts.

MAIN OUTCOME MEASURES

In the neoadjuvant treatment group, images taken before pretreatment were classified as baseline images and those taken after pretreatment as presurgery images. In the upfront surgery group, presurgery images were classified as both baseline and presurgery images. We constructed 2 types of artificial intelligence, using baseline and presurgery images, by inputting the patches from these images into ResNet-18, and we assessed their diagnostic accuracy.

RESULTS

Overall, 124 patients underwent surgery alone, 52 received neoadjuvant chemotherapy, and 33 received chemoradiotherapy. The number of resected lateral lymph nodes in the training, test, and validation cohorts was 2418, 279, and 850, respectively. The metastatic rates were 2.8%, 0.7%, and 3.7%, respectively. In the validation cohort, the precision-recall area under the curve was 0.870 and 0.963 for the baseline and presurgery images, respectively. Although both baseline and presurgery images provided good accuracy for diagnosing lateral lymph node metastases, the accuracy of presurgery images was better than that of baseline images.

LIMITATIONS

The number of cases is small.

CONCLUSIONS

An artificial intelligence tool is a promising tool for diagnosing lateral lymph node metastasis with high accuracy.

DESARROLLO DE UNA HERRAMIENTA DE INTELIGENCIA ARTIFICIAL PARA EL DIAGNSTICO DE METSTASIS EN GANGLIOS LINFTICOS LATERALES EN CNCER DE RECTO AVANZADO

ANTECEDENTES:Disección de nódulos linfáticos laterales metastásicos puede mejorar la supervivencia en pacientes con adenocarcinoma del recto, con o sin quimiorradioterapia. Sin embargo, aún no se han determinado los criterios óptimos de diagnóstico por imágenes de los nódulos linfáticos laterales metastásicos.OBJETIVO:Nuestro objetivo fue desarrollar una herramienta de inteligencia artificial para el diagnóstico de metástasis en nódulos linfáticos laterales mediante el aprendizaje profundo, para pacientes con adenocarcinoma del recto que se sometieron a cirugía radical y disección de nódulos linfáticos laterales.DISEÑO:Estudio retrospectivo.AJUSTES:Estudio multicéntrico.PACIENTES:Un total de 209 pacientes con adenocarcinoma del recto, que se sometieron a cirugía radical y disección de nódulos linfáticos laterales en 15 hospitales participantes, se inscribieron en el estudio y se asignaron a cohortes de entrenamiento (n = 139), prueba (n = 17) o validación (n = 53).PRINCIPALES MEDIDAS DE RESULTADO:En el grupo de tratamiento neoadyuvante, las imágenes tomadas antes del tratamiento se clasificaron como imágenes de referencia y las posteriores al tratamiento, como imágenes previas a la cirugía. En el grupo de cirugía inicial, las imágenes previas a la cirugía se clasificaron como imágenes de referencia y previas a la cirugía. Construimos dos tipos de inteligencia artificial, utilizando imágenes de referencia y previas a la cirugía, ingresando los parches de estas imágenes en ResNet-18. Evaluamos la precisión diagnóstica de los dos tipos de inteligencia artificial.RESULTADOS:En general, 124 pacientes se sometieron a cirugía solamente, 52 recibieron quimioterapia neoadyuvante y 33 recibieron quimiorradioterapia. El número de nódulos linfáticos laterales removidos en los cohortes de entrenamiento, prueba y validación fue de 2,418; 279 y 850, respectivamente. Las tasas metastásicas fueron 2.8%, 0.7%, y 3.7%, respectivamente. En el cohorte de validación, el área de recuperación de precisión bajo la curva fue de 0.870 y 0.963 para las imágenes de referencia y antes de la cirugía, respectivamente. Aunque tanto las imágenes previas a la cirugía como las iniciales proporcionaron una buena precisión para diagnosticar metástasis en los nódulos linfáticos laterales, la precisión de las imágenes previas a la cirugía fue mejor que la de las imágenes iniciales.LIMITACIONES:El número de casos es pequeño.CONCLUSIÓN:La inteligencia artificial es una herramienta prometedora para diagnosticar metástasis en los nódulos linfáticos laterales con alta precisión. (Traducción-Dr. Aurian Garcia Gonzalez ).

New Alternative Techniques for Sentinel Lymph Node Biopsy

Background and Objectives: This review paper highlights the key alternatives to the blue dye/radioisotope method of sentinel lymph node biopsy (SLNB). It analyses the research available on these alternative methods and their outcomes compared to the traditional techniques. Materials and Methods: This review focused on fifteen articles, of which five used indocyanine green (ICG) as a tracer, four used magnetic tracers, one used one-step nucleic acid amplification (OSNA) and Metasin (quantitative reverse transcriptase-polymerase chain reaction), one used the photosensitiser talaporfin sodium, one used sulphur hexafluoride gas microbubbles, one used CT-guided lymphography and two focused on general SLNB technique reviews. Results: Of the 15 papers analysed, the sentinel node detection rates were 69-100% for indocyanine green, 91.67-100% for magnetic tracers, 81% for talaporfin sodium, 9.3-55.2% for sulphur hexafluoride gas microbubbles, 90.5% for CTLG and 82.7-100% for one-step nucleic acid amplification. Conclusions: Indocyanine green fluorescence (ICG) and magnetic tracers have been proven non-inferior to traditional blue dye and isotope regarding SLNB localisation. Further studies are needed to investigate the use of these techniques in conjunction with each other and the possible use of language learning models. Dedicated studies are required to assess cost efficacy and longer-term outcomes.

Application of CT and MRI images based on an artificial intelligence algorithm for predicting lymph node metastasis in breast cancer patients: a meta-analysis

BACKGROUND

This study aimed to comprehensively evaluate the accuracy and effect of computed tomography (CT) and magnetic resonance imaging (MRI) based on artificial intelligence (AI) algorithms for predicting lymph node metastasis in breast cancer patients.

METHODS

We systematically searched the PubMed, Embase and Cochrane Library databases for literature from inception to June 2023 using keywords that included 'artificial intelligence', 'CT,' 'MRI', 'breast cancer' and 'lymph nodes'. Studies that met the inclusion criteria were screened and their data were extracted for analysis. The main outcome measures included sensitivity, specificity, positive likelihood ratio, negative likelihood ratio and area under the curve (AUC).

RESULTS

A total of 16 studies were included in the final meta-analysis, covering 4,764 breast cancer patients. Among them, 11 studies used the manual algorithm MRI to calculate breast cancer risk, which had a sensitivity of 0.85 (95% confidence interval [CI] 0.79-0.90; p < 0.001; I2 = 75.3%), specificity of 0.81 (95% CI 0.66-0.83; p < 0.001; I2 = 0%), a positive likelihood ratio of 4.6 (95% CI 4.0-4.8), a negative likelihood ratio of 0.18 (95% CI 0.13-0.26) and a diagnostic odds ratio of 25 (95% CI 17-38). Five studies used manual algorithm CT to calculate breast cancer risk, which had a sensitivity of 0.88 (95% CI 0.79-0.94; p < 0.001; I2 = 87.0%), specificity of 0.80 (95% CI 0.69-0.88; p < 0.001; I2 = 91.8%), a positive likelihood ratio of 4.4 (95% CI 2.7-7.0), a negative likelihood ratio of 0.15 (95% CI 0.08-0.27) and a diagnostic odds ratio of 30 (95% CI 12-72). For MRI and CT, the AUC after study pooling was 0.85 (95% CI 0.82-0.88) and 0.91 (95% CI 0.88-0.93), respectively.

CONCLUSION

Computed tomography and MRI images based on an AI algorithm have good diagnostic accuracy in predicting lymph node metastasis in breast cancer patients and have the potential for clinical application.

Artificial intelligence in breast imaging: Current situation and clinical challenges

Breast cancer ranks among the most prevalent malignant tumours and is the primary contributor to cancer-related deaths in women. Breast imaging is essential for screening, diagnosis, and therapeutic surveillance. With the increasing demand for precision medicine, the heterogeneous nature of breast cancer makes it necessary to deeply mine and rationally utilize the tremendous amount of breast imaging information. With the rapid advancement of computer science, artificial intelligence (AI) has been noted to have great advantages in processing and mining of image information. Therefore, a growing number of scholars have started to focus on and research the utility of AI in breast imaging. Here, an overview of breast imaging databases and recent advances in AI research are provided, the challenges and problems in this field are discussed, and then constructive advice is further provided for ongoing scientific developments from the perspective of the National Natural Science Foundation of China.

Prediction of axillary lymph node metastasis in early breast cancer patients with ultrasonic videos based deep learning

Objective

To develop a deep learning (DL) model for predicting axillary lymph node (ALN) metastasis using dynamic ultrasound (US) videos in breast cancer patients.

Methods

A total of 271 US videos from 271 early breast cancer patients collected from Xiang'an Hospital of Xiamen University andShantou Central Hospitabetween September 2019 and June 2021 were used as the training, validation, and internal testing set (testing set A). Additionally, an independent dataset of 49 US videos from 49 patients with breast cancer, collected from Shanghai 10th Hospital of Tongji University from July 2021 to May 2022, was used as an external testing set (testing set B). All ALN metastases were confirmed using pathological examination. Three different convolutional neural networks (CNNs) with R2 + 1D, TIN, and ResNet-3D architectures were used to build the models. The performance of the US video DL models was compared with that of US static image DL models and axillary US examination performed by ultra-sonographers. The performances of the DL models and ultra-sonographers were evaluated based on accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC). Additionally, gradient class activation mapping (Grad-CAM) technology was also used to enhance the interpretability of the models.

Results

Among the three US video DL models, TIN showed the best performance, achieving an AUC of 0.914 (95% CI: 0.843-0.985) in predicting ALN metastasis in testing set A. The model achieved an accuracy of 85.25% (52/61), with a sensitivity of 76.19% (16/21) and a specificity of 90.00% (36/40). The AUC of the US video DL model was superior to that of the US static image DL model (0.856, 95% CI: 0.753-0.959, P<0.05). The Grad-CAM technology confirmed the heatmap of the model, which highlighted important subregions of the keyframe for ultra-sonographers' review.

Conclusion

A feasible and improved DL model to predict ALN metastasis from breast cancer US video images was developed. The DL model in this study with reliable interpretability would provide an early diagnostic strategy for the appropriate management of axillary in the early breast cancer patients.

Development and validation of convolutional neural network-based model to predict the risk of sentinel or non-sentinel lymph node metastasis in patients with breast cancer: a machine learning study

Background

For patients with sentinel lymph node (SLN) metastasis and low risk of residual non-SLN (NSLN) metastasis, axillary lymph node (ALN) dissection could lead to overtreatment. This study aimed to develop and validate an automated preoperative deep learning-based tool to predict the risk of SLN and NSLN metastasis in patients with breast cancer (BC) using dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) images.

Methods

In this machine learning study, we retrospectively enrolled 988 women with BC from three hospitals in Zhejiang, China between June 1, 2013 to December 31, 2021, June 1, 2017 to December 31, 2021, and January 1, 2019 to June 30, 2023, respectively. Patients were divided into the training set (n = 519), internal validation set (n = 129), external test set 1 (n = 296), and external test set 2 (n = 44). A convolutional neural network (CNN) model was proposed to predict the SLN and NSLN metastasis and was compared with clinical and radiomics approaches. The performance of different models to detect ALN metastasis was measured by the area under the curve (AUC), accuracy, sensitivity, and specificity. This study is registered at ChiCTR, ChiCTR2300070740.

Findings

For SLN prediction, the top-performing model (i.e., the CNN algorithm) achieved encouraging predictive performance in the internal validation set (AUC 0.899, 95% CI, 0.887-0.911), external test set 1 (AUC 0.885, 95% CI, 0.867-0.903), and external test set 2 (AUC 0.768, 95% CI, 0.738-0.798). For NSLN prediction, the CNN-based model also exhibited satisfactory performance in the internal validation set (AUC 0.800, 95% CI, 0.783-0.817), external test set 1 (AUC 0.763, 95% CI, 0.732-0.794), and external test set 2 (AUC 0.728, 95% CI, 0.719-0.738). Based on the subgroup analysis, the CNN model performed well in tumour group smaller than 2.0 cm, with the AUC of 0.801 (internal validation set) and 0.823 (external test set 1). Of 469 patients with BC, the false positive rate of SLN prediction declined from 77.9% to 32.9% using CNN model.

Interpretation

The CNN model can predict the SLN status of any detectable lesion size and condition of NSLN in patients with BC. Overall, the CNN model, employing ready DCE-MRI images could serve as a potential technique to assist surgeons in the personalized axillary treatment of in patients with BC non-invasively.

Funding

National Key Research and Development projects intergovernmental cooperation in science and technology of China, National Natural Science Foundation of China, Natural Science Foundation of Zhejiang Province, and Zhejiang Medical and Health Science Project.

Artificial intelligence to support person-centred care in breast imaging - A scoping review

AIM

To overview Artificial Intelligence (AI) developments and applications in breast imaging (BI) focused on providing person-centred care in diagnosis and treatment for breast pathologies.

METHODS

The scoping review was conducted in accordance with the Joanna Briggs Institute methodology. The search was conducted on MEDLINE, Embase, CINAHL, Web of science, IEEE explore and arxiv during July 2022 and included only studies published after 2016, in French and English. Combination of keywords and Medical Subject Headings terms (MeSH) related to breast imaging and AI were used. No keywords or MeSH terms related to patients, or the person-centred care (PCC) concept were included. Three independent reviewers screened all abstracts and titles, and all eligible full-text publications during a second stage.

RESULTS

3417 results were identified by the search and 106 studies were included for meeting all criteria. Six themes relating to the AI-enabled PCC in BI were identified: individualised risk prediction/growth and prediction/false negative reduction (44.3%), treatment assessment (32.1%), tumour type prediction (11.3%), unnecessary biopsies reduction (5.7%), patients' preferences (2.8%) and other issues (3.8%). The main BI modalities explored in the included studies were magnetic resonance imaging (MRI) (31.1%), mammography (27.4%) and ultrasound (23.6%). The studies were predominantly retrospective, and some variations (age range, data source, race, medical imaging) were present in the datasets used.

CONCLUSIONS

The AI tools for person-centred care are mainly designed for risk and cancer prediction and disease management to identify the most suitable treatment. However, further studies are needed for image acquisition optimisation for different patient groups, improvement and customisation of patient experience and for communicating to patients the options and pathways of disease management.

Value of Breast MRI and Nomogram After Negative Axillary Ultrasound for Predicting Axillary Lymph Node Metastasis in Patients With Clinically T1-2 N0 Breast Cancer

BACKGROUND

There are increasing concerns about that sentinel lymph node biopsy (SLNB) could be omitted in patients with clinically T1-2 N0 breast cancers who has negative axillary ultrasound (AUS). This study aims to assess the false negative result (FNR) of AUS, the rate of high nodal burden (HNB) in clinically T1-2 N0 breast cancer patients, and the diagnostic performance of breast magnetic resonance imaging (MRI) and nomogram.

METHODS

We identified 948 consecutive patients with clinically T1-2 N0 cancers who had negative AUS, subsequent MRI, and breast conserving therapy between 2013 and 2020 from two tertiary medical centers. Patients from two centers were assigned to development and validation sets, respectively. Among 948 patients, 402 (mean age ± standard deviation, 57.61 ± 11.58) were within development cohort and 546 (54.43 ± 10.02) within validation cohort. Using logistic regression analyses, clinical-imaging factors associated with lymph node (LN) metastasis were analyzed in the development set from which nomogram was created. The performance of MRI and nomogram was assessed. HNB was defined as ≥ 3 positive LNs.

RESULTS

The FNR of AUS was 20.1% (81 of 402) and 19.2% (105 of 546) and the rates of HNB were 1.2% (5/402) and 2.2% (12/546), respectively. Clinical and imaging features associated with LN metastasis were progesterone receptor positivity, outer tumor location on mammography, breast imaging reporting and data system category 5 assessment of cancer on ultrasound, and positive axilla on MRI. In validation cohorts, the positive predictive value (PPV) and negative predictive value (NPV) of MRI and clinical-imaging nomogram was 58.5% and 86.5%, and 56.0% and 82.0%, respectively.

CONCLUSION

The FNR of AUS was approximately 20% but the rate of HNB was low. The diagnostic performance of MRI was not satisfactory with low PPV but MRI had merit in reaffirming negative AUS with high NPV. Patients who had low probability scores from our clinical-imaging nomogram might be possible candidates for the omission of SLNB.

Emerging uses of artificial intelligence in breast and axillary ultrasound

Breast ultrasound is a valuable adjunctive tool to mammography in detecting breast cancer, especially in women with dense breasts. Ultrasound also plays an important role in staging breast cancer by assessing axillary lymph nodes. However, its utility is limited by operator dependence, high recall rate, low positive predictive value and low specificity. These limitations present an opportunity for artificial intelligence (AI) to improve diagnostic performance and pioneer novel uses of ultrasound. Research in developing AI for radiology has flourished over the past few years. A subset of AI, deep learning, uses interconnected computational nodes to form a neural network, which extracts complex visual features from image data to train itself into a predictive model. This review summarizes several key studies evaluating AI programs' performance in predicting breast cancer and demonstrates that AI can assist radiologists and address limitations of ultrasound by acting as a decision support tool. This review also touches on how AI programs allow for novel predictive uses of ultrasound, particularly predicting molecular subtypes of breast cancer and response to neoadjuvant chemotherapy, which have the potential to change how breast cancer is managed by providing non-invasive prognostic and treatment data from ultrasound images. Lastly, this review explores how AI programs demonstrate improved diagnostic accuracy in predicting axillary lymph node metastasis. The limitations and future challenges in developing and implementing AI for breast and axillary ultrasound will also be discussed.

The Role of AI in Breast Cancer Lymph Node Classification: A Comprehensive Review

Breast cancer is a significant health issue affecting women worldwide, and accurately detecting lymph node metastasis is critical in determining treatment and prognosis. While traditional diagnostic methods have limitations and complications, artificial intelligence (AI) techniques such as machine learning (ML) and deep learning (DL) offer promising solutions for improving and supplementing diagnostic procedures. Current research has explored state-of-the-art DL models for breast cancer lymph node classification from radiological images, achieving high performances (AUC: 0.71-0.99). AI models trained on clinicopathological features also show promise in predicting metastasis status (AUC: 0.74-0.77), whereas multimodal (radiomics + clinicopathological features) models combine the best from both approaches and also achieve good results (AUC: 0.82-0.94). Once properly validated, such models could greatly improve cancer care, especially in areas with limited medical resources. This comprehensive review aims to compile knowledge about state-of-the-art AI models used for breast cancer lymph node metastasis detection, discusses proper validation techniques and potential pitfalls and limitations, and presents future directions and best practices to achieve high usability in real-world clinical settings.

Clinical applications of deep learning in breast MRI

Deep learning (DL) is one of the most powerful data-driven machine-learning techniques in artificial intelligence (AI). It can automatically learn from raw data without manual feature selection. DL models have led to remarkable advances in data extraction and analysis for medical imaging. Magnetic resonance imaging (MRI) has proven useful in delineating the characteristics and extent of breast lesions and tumors. This review summarizes the current state-of-the-art applications of DL models in breast MRI. Many recent DL models were examined in this field, along with several advanced learning approaches and methods for data normalization and breast and lesion segmentation. For clinical applications, DL-based breast MRI models were proven useful in five aspects: diagnosis of breast cancer, classification of molecular types, classification of histopathological types, prediction of neoadjuvant chemotherapy response, and prediction of lymph node metastasis. For subsequent studies, further improvement in data acquisition and preprocessing is necessary, additional DL techniques in breast MRI should be investigated, and wider clinical applications need to be explored.

Predicting of axillary lymph node metastasis in invasive breast cancer using multiparametric MRI dataset based on CNN model

Purpose

To develop a multiparametric MRI model for predicting axillary lymph node metastasis in invasive breast cancer.

Methods

Clinical data and T2WI, DWI, and DCE-MRI images of 252 patients with invasive breast cancer were retrospectively analyzed and divided into the axillary lymph node metastasis (ALNM) group and non-ALNM group using biopsy results as a reference standard. The regions of interest (ROI) in T2WI, DWI, and DCE-MRI images were segmented using MATLAB software, and the ROI was unified into 224 × 224 sizes, followed by image normalization as input to T2WI, DWI, and DCE-MRI models, all of which were based on ResNet 50 networks. The idea of a weighted voting method in ensemble learning was employed, and then T2WI, DWI, and DCE-MRI models were used as the base models to construct a multiparametric MRI model. The entire dataset was randomly divided into training sets and testing sets (the training set 202 cases, including 78 ALNM, 124 non-ALNM; the testing set 50 cases, including 20 ALNM, 30 non-ALNM). Accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of models were calculated. The receiver operating characteristic (ROC) curve and area under the curve (AUC) were used to evaluate the diagnostic performance of each model for axillary lymph node metastasis, and the DeLong test was performed, P< 0.05 statistically significant.

Results

For the assessment of axillary lymph node status in invasive breast cancer on the test set, multiparametric MRI models yielded an AUC of 0.913 (95% CI, 0.799-0.974); T2WI-based model yielded an AUC of 0.908 (95% CI, 0.792-0.971); DWI-based model achieved an AUC of 0.702 (95% CI, 0.556-0.823); and the AUC of the DCE-MRI-based model was 0.572 (95% CI, 0.424-0.711). The improvement in the diagnostic performance of the multiparametric MRI model compared with the DWI and DCE-MRI-based models were significant (P< 0.01 for both). However, the increase was not meaningful compared with the T2WI-based model (P = 0.917).

Conclusion

Multiparametric MRI image analysis based on an ensemble CNN model with deep learning is of practical application and extension for preoperative prediction of axillary lymph node metastasis in invasive breast cancer.

Deep Learning Prediction of Pathologic Complete Response in Breast Cancer Using MRI and Other Clinical Data: A Systematic Review

Breast cancer patients who have pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) are more likely to have better clinical outcomes. The ability to predict which patient will respond to NAC early in the treatment course is important because it could help to minimize unnecessary toxic NAC and to modify regimens mid-treatment to achieve better efficacy. Machine learning (ML) is increasingly being used in radiology and medicine because it can identify relationships amongst complex data elements to inform outcomes without the need to specify such relationships a priori. One of the most popular deep learning methods that applies to medical images is the Convolutional Neural Networks (CNN). In contrast to supervised ML, deep learning CNN can operate on the whole images without requiring radiologists to manually contour the tumor on images. Although there have been many review papers on supervised ML prediction of pCR, review papers on deep learning prediction of pCR are sparse. Deep learning CNN could also incorporate multiple image types, clinical data such as demographics and molecular subtypes, as well as data from multiple treatment time points to predict pCR. The goal of this study is to perform a systematic review of deep learning methods that use whole-breast MRI images without annotation or tumor segmentation to predict pCR in breast cancer.

A nomogram based on radiomics signature and deep-learning signature for preoperative prediction of axillary lymph node metastasis in breast cancer

PURPOSE

To develop a nomogram based on radiomics signature and deep-learning signature for predicting the axillary lymph node (ALN) metastasis in breast cancer.

METHODS

A total of 151 patients were assigned to a training cohort (n = 106) and a test cohort (n = 45) in this study. Radiomics features were extracted from DCE-MRI images, and deep-learning features were extracted by VGG-16 algorithm. Seven machine learning models were built using the selected features to evaluate the predictive value of radiomics or deep-learning features for the ALN metastasis in breast cancer. A nomogram was then constructed based on the multivariate logistic regression model incorporating radiomics signature, deep-learning signature, and clinical risk factors.

RESULTS

Five radiomics features and two deep-learning features were selected for machine learning model construction. In the test cohort, the AUC was above 0.80 for most of the radiomics models except DecisionTree and ExtraTrees. In addition, the K-nearest neighbor (KNN), XGBoost, and LightGBM models using deep-learning features had AUCs above 0.80 in the test cohort. The nomogram, which incorporated the radiomics signature, deep-learning signature, and MRI-reported LN status, showed good calibration and performance with the AUC of 0.90 (0.85-0.96) in the training cohort and 0.90 (0.80-0.99) in the test cohort. The DCA showed that the nomogram could offer more net benefit than radiomics signature or deep-learning signature.

CONCLUSIONS

Both radiomics and deep-learning features are diagnostic for predicting ALN metastasis in breast cancer. The nomogram incorporating radiomics and deep-learning signatures can achieve better prediction performance than every signature used alone.

Artificial Intelligence in Breast Ultrasound: The Emerging Future of Modern Medicine

In today's world, progressively enormous popularity prevails around artificial intelligence (AI). AI is gaining popularity in the identification of various images. Therefore, it has been widely used in the ultrasound of the breast. Furthermore, AI can perform a quantitative evaluation, which further helps maintain the diagnosis's accuracy. Moreover, breast cancer is the most common cancer in women, posing a severe threat to women's health. Hence, its early detection is usually associated with a patient's prognosis. As a result, using AI in breast cancer screening and detection is highly crucial. The concept of AI in the perspective of breast ultrasound has been highlighted in this brief review article. It tends to focus on early AI, i.e., traditional machine learning and deep learning algorithms. Also, the use of AI in ultrasound and the use of it in mammography, magnetic resonance imaging, nuclear medicine imaging, and classification of breast lesions is broadly explained, along with the challenges faced in bringing AI into daily practice.

Preoperative Prediction of Axillary Lymph Node Metastasis in Breast Cancer Using CNN Based on Multiparametric MRI

BACKGROUND

Multiparametric magnetic resonance imaging (MRI) is widely used in breast cancer screening. Accurate prediction of the axillary lymph nodes metastasis (ALNM) is essential for breast cancer surgery and treatment. However, there is no mature and effective discerning method for ALNM based on multiparametric MRI.

PURPOSE

To evaluate the ALNM using T1-weighted imaging (T1WI), T2-weighted imaging (T2WI), and diffusion-weighted imaging (DWI) sequences, respectively, and construct a quantitative ALNM discerning model of integrated multiparametric MRI.

STUDY TYPE

Retrospective.

POPULATION

Three-hundred forty-eight breast cancer patients, 163 with ALNM (99.39% females), and 185 without ALNM (100% females). The dataset was randomly divided into the training set (315 cases) and the testing set (33 cases).

FIELD STRENGTH/SEQUENCE

1.5 T; T1WI (VIBRANT), T2WI (FSE), and DWI (echo planar imaging [EPI]).

ASSESSMENT

The lesion region of interest images were cropped and sent to a pretrained ResNet50 network. Then, the results of different sequences were sent to a classifier for ensemble learning to construct the ALNM model of multiparametric MRI.

STATISTICAL TESTS

Performance indicators such as accuracy, the receiver operating characteristic (ROC) curve, and the area under the ROC curve (AUC) were calculated. Student's t-test, chi-square test, Fisher's exact test, and Delong test were performed, and P < 0.05 was considered statistically significant.

RESULTS

T2WI performed the best among the three sequences, and achieved the accuracy and AUC of 0.933/0.989 in the testing set. Compared to T1WI with the accuracy and AUC of 0.691/0.806, the increase is significant. While compared to DWI with the accuracy and AUC of 0.800/0.910, the improvement is not significant (P = 0.126). After integrating three sequences, the accuracy and AUC improved to 0.970 and 0.996.

DATA CONCLUSION

T2WI performed better than DWI and T1WI in discerning ALNM in this breast cancer dataset. The proposed quantitative model of integrated multiparametric MRI could effectively help the ALNM diagnosis.

LEVEL OF EVIDENCE

1 TECHNICAL EFFICACY STAGE: 2.

Detecting Abnormal Axillary Lymph Nodes on Mammograms Using a Deep Convolutional Neural Network

The purpose of this study was to determine the feasibility of a deep convolutional neural network (dCNN) to accurately detect abnormal axillary lymph nodes on mammograms. In this retrospective study, 107 mammographic images in mediolateral oblique projection from 74 patients were labeled to three classes: (1) “breast tissue”, (2) “benign lymph nodes”, and (3) “suspicious lymph nodes”. Following data preprocessing, a dCNN model was trained and validated with 5385 images. Subsequently, the trained dCNN was tested on a “real-world” dataset and the performance compared to human readers. For visualization, colored probability maps of the classification were calculated using a sliding window approach. The accuracy was 98% for the training and 99% for the validation set. Confusion matrices of the “real-world” dataset for the three classes with radiological reports as ground truth yielded an accuracy of 98.51% for breast tissue, 98.63% for benign lymph nodes, and 95.96% for suspicious lymph nodes. Intraclass correlation of the dCNN and the readers was excellent (0.98), and Kappa values were nearly perfect (0.93–0.97). The colormaps successfully detected abnormal lymph nodes with excellent image quality. In this proof-of-principle study in a small patient cohort from a single institution, we found that deep convolutional networks can be trained with high accuracy and reliability to detect abnormal axillary lymph nodes on mammograms.

Deep learning in breast imaging

Millions of breast imaging exams are performed each year in an effort to reduce the morbidity and mortality of breast cancer. Breast imaging exams are performed for cancer screening, diagnostic work-up of suspicious findings, evaluating extent of disease in recently diagnosed breast cancer patients, and determining treatment response. Yet, the interpretation of breast imaging can be subjective, tedious, time-consuming, and prone to human error. Retrospective and small reader studies suggest that deep learning (DL) has great potential to perform medical imaging tasks at or above human-level performance, and may be used to automate aspects of the breast cancer screening process, improve cancer detection rates, decrease unnecessary callbacks and biopsies, optimize patient risk assessment, and open up new possibilities for disease prognostication. Prospective trials are urgently needed to validate these proposed tools, paving the way for real-world clinical use. New regulatory frameworks must also be developed to address the unique ethical, medicolegal, and quality control issues that DL algorithms present. In this article, we review the basics of DL, describe recent DL breast imaging applications including cancer detection and risk prediction, and discuss the challenges and future directions of artificial intelligence-based systems in the field of breast cancer.

Deep learning prediction of axillary lymph node status using ultrasound images

OBJECTIVE

To investigate the ability of our convolutional neural network (CNN) to predict axillary lymph node metastasis using primary breast cancer ultrasound (US) images.

METHODS

In this IRB-approved study, 338 US images (two orthogonal images) from 169 patients from 1/2014-12/2016 were used. Suspicious lymph nodes were seen on US and patients subsequently underwent core-biopsy. 64 patients had metastatic lymph nodes. A custom CNN was utilized on 248 US images from 124 patients in the training dataset and tested on 90 US images from 45 patients. The CNN was implemented entirely of 3 × 3 convolutional kernels and linear layers. The 9 convolutional kernels consisted of 6 residual layers, totaling 12 convolutional layers. Feature maps were down-sampled using strided convolutions. Dropout with a 0.5 keep probability and L2 normalization was utilized. Training was implemented by using the Adam optimizer and a final SoftMax score threshold of 0.5 from the average of raw logits from each pixel was used for two class classification (metastasis or not).

RESULTS

Our CNN achieved an AUC of 0.72 (SD ± 0.08) in predicting axillary lymph node metastasis from US images in the testing dataset. The model had an accuracy of 72.6% (SD ± 8.4) with a sensitivity and specificity of 65.5% (SD ± 28.6) and 78.9% (SD ± 15.1) respectively. Our algorithm is available to be shared for research use. (https://github.com/stmutasa/MetUS).

CONCLUSION

It's feasible to predict axillary lymph node metastasis from US images using a deep learning technique. This can potentially aid nodal staging in patients with breast cancer.

Application of Deep Learning in Breast Cancer Imaging

This review gives an overview of the current state of deep learning research in breast cancer imaging. Breast imaging plays a major role in detecting breast cancer at an earlier stage, as well as monitoring and evaluating breast cancer during treatment. The most commonly used modalities for breast imaging are digital mammography, digital breast tomosynthesis, ultrasound and magnetic resonance imaging. Nuclear medicine imaging techniques are used for detection and classification of axillary lymph nodes and distant staging in breast cancer imaging. All of these techniques are currently digitized, enabling the possibility to implement deep learning (DL), a subset of Artificial intelligence, in breast imaging. DL is nowadays embedded in a plethora of different tasks, such as lesion classification and segmentation, image reconstruction and generation, cancer risk prediction, and prediction and assessment of therapy response. Studies show similar and even better performances of DL algorithms compared to radiologists, although it is clear that large trials are needed, especially for ultrasound and magnetic resonance imaging, to exactly determine the added value of DL in breast cancer imaging. Studies on DL in nuclear medicine techniques are only sparsely available and further research is mandatory. Legal and ethical issues need to be considered before the role of DL can expand to its full potential in clinical breast care practice.

Global-Local attention network with multi-task uncertainty loss for abnormal lymph node detection in MR images

Accurate and reliable detection of abnormal lymph nodes in magnetic resonance (MR) images is very helpful for the diagnosis and treatment of numerous diseases. However, it is still a challenging task due to similar appearances between abnormal lymph nodes and other tissues. In this paper, we propose a novel network based on an improved Mask R-CNN framework for the detection of abnormal lymph nodes in MR images. Instead of laboriously collecting large-scale pixel-wise annotated training data, pseudo masks generated from RECIST bookmarks on hand are utilized as the supervision. Different from the standard Mask R-CNN architecture, there are two main innovations in our proposed network: 1) global-local attention which encodes the global and local scale context for detection and utilizes the channel attention mechanism to extract more discriminative features and 2) multi-task uncertainty loss which adaptively weights multiple objective loss functions based on the uncertainty of each task to automatically search the optimal solution. For the experiments, we built a new abnormal lymph node dataset with 821 RECIST bookmarks of 41 different types of abnormal abdominal lymph nodes from 584 different patients. The experimental results showed the superior performance of our algorithm over compared state-of-the-art approaches.

Weakly Supervised Deep Learning Approach to Breast MRI Assessment

RATIONALE AND OBJECTIVES

To evaluate a weakly supervised deep learning approach to breast Magnetic Resonance Imaging (MRI) assessment without pixel level segmentation in order to improve the specificity of breast MRI lesion classification.

MATERIALS AND METHODS

In this IRB approved study, the dataset consisted of 278,685 image slices from 438 patients. The weakly supervised network was based on the Resnet-101 architecture. Training was implemented using the Adam optimizer and a final SoftMax score threshold of 0.5 was used for two class classification (malignant or benign). 278,685 image slices were combined into 92,895 3-channel images. 79,871 (85%) images were used for training and validation while 13,024 (15%) images were separated for testing. Of the testing dataset, 11,498 (88%) were benign and 1531 (12%) were malignant. Model performance was assessed.

RESULTS

The weakly supervised network achieved an AUC of 0.92 (SD ± 0.03) in distinguishing malignant from benign images. The model had an accuracy of 94.2% (SD ± 3.4) with a sensitivity and specificity of 74.4% (SD ± 8.5) and 95.3% (SD ± 3.3) respectively.

CONCLUSION

It is feasible to use a weakly supervised deep learning approach to assess breast MRI images without the need for pixel-by-pixel segmentation yielding a high degree of specificity in lesion classification.

A meta-analysis of the diagnostic performance of machine learning-based MRI in the prediction of axillary lymph node metastasis in breast cancer patients

Background

Despite that machine learning (ML)-based MRI has been evaluated for diagnosis of axillary lymph node metastasis (ALNM) in breast cancer patients, diagnostic values they showed have been variable. In this study, we aimed to assess the use of ML to classify ALNM on MRI and to identify potential covariates that might influence the diagnostic performance of ML.

Methods

A systematic research of PubMed, Embase, Web of Science, and the Cochrane Library was conducted until 27 December 2020 to collect the included articles. Subgroup analysis was also performed.

Findings

Fourteen studies assessing a total of 2247 breast cancer patients were included in the analysis. The overall AUC for ML in the validation set was 0.80 (95% confidence interval [CI] 0.76–0.83) with a negative predictive value of 0.83. The pooled sensitivity and specificity were 0.79 (95% CI 0.74–0.84) and 0.77 (95% CI 0.73–0.81), respectively. In the subgroup analysis of the validation set, T1-weighted contrast-enhanced (T1CE) imaging with ML yielded a higher sensitivity (0.80 vs. 0.67 vs. 0.76) than the T2-weighted fat-suppressed (T2-FS) imaging and diffusion-weighted imaging (DWI). Support vector machines (SVMs) had a higher specificity than linear regression (LR) and linear discriminant analysis (LDA) (0.79 vs. 0.78 vs. 0.75), whereas LDA showed a higher sensitivity than LR and SVM (0.83 vs. 0.70 vs. 0.77).

Interpretation

MRI sequences and algorithms were the main factors that affect the diagnostic performance of ML. Although its results were encouraging with the pooled sensitivity of around 0.80, it meant that 1 in 5 women that would go with undetected metastases, which may have a detrimental effect on the overall survival for 20% of patients with positive SLN status. Despite that a high NPV of 0.83 meant that ML could potentially benefit those with negative SLN, it might also translate to 1 in 5 tests being false negative. We would like to suggest that ML may not be yet usable in clinical routine especially when patient survival is used as a primary measurement of its outcome.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13244-021-01034-1.

Machine learning for evolutive lymphoma and residual masses recognition in whole body diffusion weighted magnetic resonance images

BACKGROUND

After the treatment of the patients with malignant lymphoma, there may persist lesions that must be labeled either as evolutive lymphoma requiring new treatments or as residual masses. We present in this work, a machine learning-based computer-aided diagnosis (CAD) applied to whole-body diffusion-weighted magnetic resonance images.

METHODS

The database consists of a total of 1005 MRI images with evolutive lymphoma and residual masses. More specifically, we propose a novel approach that leverages: (1)-The complementarity of the functional and anatomical criteria of MRI images through a fusion step based on the discrete wavelet transforms (DWT). (2)- The automatic segmentation of the lesions, their localization, and their enumeration using the Chan-Vese algorithm. (3)- The generation of the parametric image which contains the apparent diffusion coefficient value named ADC map. (4)- The features selection through the application of the sequential forward selection (SFS), Entropy, Symmetric uncertainty and Gain Ratio algorithm on 72 extracted features. (5)- The classification of the lesions by applying five well known supervised machine learning classification algorithms: the back-propagation artificial neural network (ANN), the support vector machine (SVM), the K-nearest neighbours (K-NN), Relevance Vectors Machine (RVM), and the random forest (RF) compared to deep learning based on convolutional neural network (CNN). Moreover, this study is achieved with an evaluation of the classification using 335 DW-MR images where 80% of them are used for the training and the remaining 20% for the test.

RESULTS

The obtained accuracy for the five classifiers recorded a slight superiority to the proposed method based on the back-propagation 3-9-1 ANN model which reaches 96,5%. In addition, we compared the proposed method to five other works from the literature. The proposed method gives much better results in terms of SE, SP, accuracy, F₁-measure, and geometric-mean which reaches respectively 96.4%, 90.9%, 95.5%, 0.97, and 91.61%.

CONCLUSIONS

Our initial results suggest that Combining functional, anatomical, and morphological features of ROI's have very good accuracy (97.01%) for evolutive lymphoma and residual masses recognition when we based on the new proposed approach using the back-propagation 3-9-1 ANN model. Proposed method based on machine learning gives less than Deep learning CNN, which is 98.5%.

Development of MRI-based axillary numerical models and estimation of axillary lymph node dielectric properties for microwave imaging

PURPOSE

Microwave imaging (MWI) has been studied as a complementary imaging modality to improve sensitivity and specificity of diagnosis of axillary lymph nodes (ALNs), which can be metastasized by breast cancer. The feasibility of such a system is based on the dielectric contrast between healthy and metastasized ALNs. However, reliable information such as anatomically realistic numerical models and matching dielectric properties of the axillary region and ALNs, which are crucial to develop MWI systems, are still limited in the literature. The purpose of this work is to develop a methodology to infer dielectric properties of structures from magnetic resonance imaging (MRI), in particular, ALNs. We further use this methodology, which is tailored for structures farther away from MR coils, to create MRI-based numerical models of the axillary region and share them with the scientific community, through an open-access repository.

METHODS

We use a dataset of breast MRI scans of 40 patients, 15 of them with metastasized ALNs. We apply image processing techniques to minimize the artifacts in MR images and segment the tissues of interest. The background, lung cavity, and skin are segmented using thresholding techniques and the remaining tissues are segmented using a K-means clustering algorithm. The ALNs are segmented combining the clustering results of two MRI sequences. The performance of this methodology was evaluated using qualitative criteria. We then apply a piecewise linear interpolation between voxel signal intensities and known dielectric properties, which allow us to create dielectric property maps within an MRI and consequently infer ALN properties. Finally, we compare healthy and metastasized ALN dielectric properties within and between patients, and we create an open-access repository of numerical axillary region numerical models which can be used for electromagnetic simulations.

RESULTS

The proposed methodology allowed creating anatomically realistic models of the axillary region, segmenting 80 ALNs and analyzing the corresponding dielectric properties. The estimated relative permittivity of those ALNs ranged from 16.6 to 49.3 at 5 GHz. We observe there is a high variability of dielectric properties of ALNs, which can be mainly related to the ALN size and, consequently, its composition. We verified an average dielectric contrast of 29% between healthy and metastasized ALNs. Our repository comprises 10 numerical models of the axillary region, from five patients, with variable number of metastasized ALNs and body mass index.

CONCLUSIONS

The observed contrast between healthy and metastasized ALNs is a good indicator for the feasibility of a MWI system aiming to diagnose ALNs. This paper presents new contributions regarding anatomical modeling and dielectric properties' characterization, in particular for axillary region applications.

MRI-based radiomics analysis for predicting the EGFR mutation based on thoracic spinal metastases in lung adenocarcinoma patients

PURPOSE

This study aims to develop and evaluate multi-parametric MRI-based radiomics for preoperative identification of epidermal growth factor receptor (EGFR) mutation, which is important in treatment planning for patients with thoracic spinal metastases from primary lung adenocarcinoma.

METHODS

A total of 110 patients were enrolled between January 2016 and March 2019 as a primary cohort. A time-independent validation cohort was conducted containing 52 patients consecutively enrolled from July 2019 to April 2021. The patients were pathologically diagnosed with thoracic spinal metastases from primary lung adenocarcinoma; all underwent T1-weighted (T1W), T2-weighted (T2W), and T2-weighted fat-suppressed (T2FS) MRI scans of the thoracic spinal. Handcrafted and deep learning-based features were extracted and selected from each MRI modality, and used to build the radiomics signature. Various machine learning classifiers were developed and compared. A clinical-radiomics nomogram integrating the combined rad signature and the most important clinical factor was constructed with receiver operating characteristic (ROC), calibration, and decision curves analysis (DCA) to evaluate the prediction performance.

RESULTS

The combined radiomics signature derived from the joint of three modalities can effectively classify EGFR mutation and EGFR wild-type patients, with an area under the ROC curve (AUC) of 0.886 (95% confidence interval [CI]: 0.826-0.947, SEN =0.935, SPE =0.688) in the training group and 0.803 (95% CI: 0.682-0.924, SEN = 0.700, SPE = 0.818) in the time-independent validation group. The nomogram incorporating the combined radiomics signature and smoking status achieved the best prediction performance in the training (AUC = 0.888, 95% CI: 0.849-0.958, SEN = 0.839, SPE = 0.792) and time-independent validation (AUC = 0.821, 95% CI: 0.692-0.929, SEN = 0.667, SPE = 0.909) cohorts. The DCA confirmed potential clinical usefulness of our nomogram.

CONCLUSION

Our study demonstrated the potential of multi-parametric MRI-based radiomics on preoperatively predicting the EGFR mutation. The proposed nomogram model can be considered as a new biomarker to guide the selection of individual treatment strategies for patients with thoracic spinal metastases from primary lung adenocarcinoma.

Convolutional Neural Network of Multiparametric MRI Accurately Detects Axillary Lymph Node Metastasis in Breast Cancer Patients With Pre Neoadjuvant Chemotherapy

BACKGROUND

Accurate assessment of the axillary lymph nodes (aLNs) in breast cancer patients is essential for prognosis and treatment planning. Current radiological staging of nodal metastasis has poor accuracy. This study aimed to investigate the machine learning convolutional neural networks (CNNs) on multiparametric MRI to detect nodal metastasis with 18FDG-PET as ground truths.

MATERIALS AND METHODS

Data were obtained via a retrospective search. Inclusion criteria were patients with bilateral breast MRI and 18FDG-PETand/or CT scans obtained before neoadjuvant chemotherapy. In total, 238 aLNs were obtained from 56 breast cancer patients with 18FDG-PET and/or CT and breast MRI data. Radiologists scored each node based on all MRI as diseased and non-diseased nodes. Five models were built using T1-W MRI, T2-W MRI, DCE MRI, T1-W + T2-W MRI, and DCE + T2-W MRI model. Performance was evaluated using receiver operating curve (ROC) analysis, including area under the curve (AUC).

RESULTS

All CNN models yielded similar performance with an accuracy ranging from 86.08% to 88.50% and AUC ranging from 0.804 to 0.882. The CNN model using T1-W MRI performed better than that using T2-W MRI in detecting nodal metastasis. CNN model using combined T1- and T2-W MRI performed the best compared to all other models (accuracy = 88.50%, AUC = 0.882), but similar in AUC to the DCE + T2-W MRI model (accuracy = 88.02%, AUC = 0.880). All CNN models performed better than radiologists in detecting nodal metastasis (accuracy = 65.8%).

CONCLUSION

xxxxxx.

Understanding artificial intelligence based radiology studies: CNN architecture

Artificial intelligence (AI) in radiology has gained wide interest due to the development of neural network architectures with high performance in computer vision related tasks. As AI based software programs become more integrated into the clinical workflow, radiologists can benefit from better understanding the principles of artificial intelligence. This series aims to explain basic concepts of AI and its applications in medical imaging. In this article, we will review the background of neural network architecture and its application in imaging analysis.

Using Deep Learning to Improve Nonsystematic Viewing of Breast Cancer on MRI

OBJECTIVE

To investigate the feasibility of using deep learning to identify tumor-containing axial slices on breast MRI images.

METHODS

This IRB-approved retrospective study included consecutive patients with operable invasive breast cancer undergoing pretreatment breast MRI between January 1, 2014, and December 31, 2017. Axial tumor-containing slices from the first postcontrast phase were extracted. Each axial image was subdivided into two subimages: one of the ipsilateral cancer-containing breast and one of the contralateral healthy breast. Cases were randomly divided into training, validation, and testing sets. A convolutional neural network was trained to classify subimages into "cancer" and "no cancer" categories. Accuracy, sensitivity, and specificity of the classification system were determined using pathology as the reference standard. A two-reader study was performed to measure the time savings of the deep learning algorithm using descriptive statistics.

RESULTS

Two hundred and seventy-three patients with unilateral breast cancer met study criteria. On the held-out test set, accuracy of the deep learning system for tumor detection was 92.8% (648/706; 95% confidence interval: 89.7%-93.8%). Sensitivity and specificity were 89.5% and 94.3%, respectively. Readers spent 3 to 45 seconds to scroll to the tumor-containing slices without use of the deep learning algorithm.

CONCLUSION

In breast MR exams containing breast cancer, deep learning can be used to identify the tumor-containing slices. This technology may be integrated into the picture archiving and communication system to bypass scrolling when viewing stacked images, which can be helpful during nonsystematic image viewing, such as during interdisciplinary tumor board meetings.

Artificial intelligence performance in detecting tumor metastasis from medical radiology imaging: A systematic review and meta-analysis

BACKGROUND

Early diagnosis of tumor metastasis is crucial for clinical treatment. Artificial intelligence (AI) has shown great promise in the field of medicine. We therefore aimed to evaluate the diagnostic accuracy of AI algorithms in detecting tumor metastasis using medical radiology imaging.

METHODS

We searched PubMed and Web of Science for studies published from January 1, 1997, to January 30, 2020. Studies evaluating an AI model for the diagnosis of tumor metastasis from medical images were included. We excluded studies that used histopathology images or medical wave-form data and those focused on the region segmentation of interest. Studies providing enough information to construct contingency tables were included in a meta-analysis.

FINDINGS

We identified 2620 studies, of which 69 were included. Among them, 34 studies were included in a meta-analysis with a pooled sensitivity of 82% (95% CI 79-84%), specificity of 84% (82-87%) and AUC of 0·90 (0·87-0·92). Analysis for different AI algorithms showed a pooled sensitivity of 87% (83-90%) for machine learning and 86% (82-89%) for deep learning, and a pooled specificity of 89% (82-93%) for machine learning, and 87% (82-91%) for deep learning.

INTERPRETATION

AI algorithms may be used for the diagnosis of tumor metastasis using medical radiology imaging with equivalent or even better performance to health-care professionals, in terms of sensitivity and specificity. At the same time, rigorous reporting standards with external validation and comparison to health-care professionals are urgently needed for AI application in the medical field.

FUNDING

College students' innovative entrepreneurial training plan program .

Understanding artificial intelligence based radiology studies: What is overfitting?

Artificial intelligence (AI) is a broad umbrella term used to encompass a wide variety of subfields dedicated to creating algorithms to perform tasks that mimic human intelligence. As AI development grows closer to clinical integration, radiologists will need to become familiar with the principles of artificial intelligence to properly evaluate and use this powerful tool. This series aims to explain certain basic concepts of artificial intelligence, and their applications in medical imaging starting with a concept of overfitting.

Breast dynamic contrast-enhanced-magnetic resonance imaging and radiomics: State of art

Breast cancer represents the most common malignancy in women, being one of the most frequent cause of cancer-related mortality. Ultrasound, mammography, and magnetic resonance imaging (MRI) play a pivotal role in the diagnosis of breast lesions, with different levels of accuracy. Particularly, dynamic contrast-enhanced MRI has shown high diagnostic value in detecting multifocal, multicentric, or contralateral breast cancers. Radiomics is emerging as a promising tool for quantitative tumor evaluation, allowing the extraction of additional quantitative data from radiological imaging acquired with different modalities. Radiomics analysis may provide novel information through the quantification of lesions heterogeneity, that may be relevant in clinical practice for the characterization of breast lesions, prediction of tumor response to systemic therapies and evaluation of prognosis in patients with breast cancers. Several published studies have explored the value of radiomics with good-to-excellent diagnostic and prognostic performances for the evaluation of breast lesions. Particularly, the integrations of radiomics data with other clinical and histopathological parameters have demonstrated to improve the prediction of tumor aggressiveness with high accuracy and provided precise models that will help to guide clinical decisions and patients management. The purpose of this article in to describe the current application of radiomics in breast dynamic contrast-enhanced MRI.

Fully Automated Postlumpectomy Breast Margin Assessment Utilizing Convolutional Neural Network Based Optical Coherence Tomography Image Classification Method

BACKGROUND

The purpose of this study was to develop a deep learning classification approach to distinguish cancerous from noncancerous regions within optical coherence tomography (OCT) images of breast tissue for potential use in an intraoperative setting for margin assessment.

METHODS

A custom ultrahigh-resolution OCT (UHR-OCT) system with an axial resolution of 2.7 μm and a lateral resolution of 5.5 μm was used in this study. The algorithm used an A-scan-based classification scheme and the convolutional neural network (CNN) was implemented using an 11-layer architecture consisting of serial 3 × 3 convolution kernels. Four tissue types were classified, including adipose, stroma, ductal carcinoma in situ, and invasive ductal carcinoma.

RESULTS

The binary classification of cancer versus noncancer with the proposed CNN achieved 94% accuracy, 96% sensitivity, and 92% specificity. The mean five-fold validation F1 score was highest for invasive ductal carcinoma (mean standard deviation, 0.89 ± 0.09) and adipose (0.79 ± 0.17), followed by stroma (0.74 ± 0.18), and ductal carcinoma in situ (0.65 ± 0.15).

CONCLUSION

It is feasible to use CNN based algorithm to accurately distinguish cancerous regions in OCT images. This fully automated method can overcome limitations of manual interpretation including interobserver variability and speed of interpretation and may enable real-time intraoperative margin assessment.

Axillary Nodal Evaluation in Breast Cancer: State of the Art

Axillary lymph node (LN) metastasis is the most important predictor of overall recurrence and survival in patients with breast cancer, and accurate assessment of axillary LN involvement is an essential component in staging breast cancer. Axillary management in patients with breast cancer has become much less invasive and individualized with the introduction of sentinel LN biopsy (SLNB). Emerging evidence indicates that axillary LN dissection may be avoided in selected patients with node-positive as well as node-negative cancer. Thus, assessment of nodal disease burden to guide multidisciplinary treatment decision making is now considered to be a critical role of axillary imaging and can be achieved with axillary US, MRI, and US-guided biopsy. For the node-positive patients treated with neoadjuvant chemotherapy, restaging of the axilla with US and MRI and targeted axillary dissection in addition to SLNB is highly recommended to minimize the false-negative rate of SLNB. Efforts continue to develop prediction models that incorporate imaging features to predict nodal disease burden and to select proper candidates for SLNB. As methods of axillary nodal evaluation evolve, breast radiologists and surgeons must work closely to maximize the potential role of imaging and to provide the most optimized treatment for patients.

Prostate Cancer Nodal Staging: Using Deep Learning to Predict 68Ga-PSMA-Positivity from CT Imaging Alone

Lymphatic spread determines treatment decisions in prostate cancer (PCa) patients. 68Ga-PSMA-PET/CT can be performed, although cost remains high and availability is limited. Therefore, computed tomography (CT) continues to be the most used modality for PCa staging. We assessed if convolutional neural networks (CNNs) can be trained to determine 68Ga-PSMA-PET/CT-lymph node status from CT alone. In 549 patients with 68Ga-PSMA PET/CT imaging, 2616 lymph nodes were segmented. Using PET as a reference standard, three CNNs were trained. Training sets balanced for infiltration status, lymph node location and additionally, masked images, were used for training. CNNs were evaluated using a separate test set and performance was compared to radiologists' assessments and random forest classifiers. Heatmaps maps were used to identify the performance determining image regions. The CNNs performed with an Area-Under-the-Curve of 0.95 (status balanced) and 0.86 (location balanced, masked), compared to an AUC of 0.81 of experienced radiologists. Interestingly, CNNs used anatomical surroundings to increase their performance, "learning" the infiltration probabilities of anatomical locations. In conclusion, CNNs have the potential to build a well performing CT-based biomarker for lymph node metastases in PCa, with different types of class balancing strongly affecting CNN performance.

Convolutional Neural Network Detection of Axillary Lymph Node Metastasis Using Standard Clinical Breast MRI

BACKGROUND

Axillary lymph node status is important for breast cancer staging and treatment planning as the majority of breast cancer metastasis spreads through the axillary lymph nodes. There is currently no reliable noninvasive imaging method to detect nodal metastasis associated with breast cancer.

MATERIALS AND METHODS

Magnetic resonance imaging (MRI) data were those from the peak contrast dynamic image from 1.5 Tesla MRI scanners at the pre-neoadjuvant chemotherapy stage. Data consisted of 66 abnormal nodes from 38 patients and 193 normal nodes from 61 patients. Abnormal nodes were those determined by expert radiologist based on ¹⁸Fluorodeoxyglucose positron emission tomography images. Normal nodes were those with negative diagnosis of breast cancer. The convolutional neural network consisted of 5 convolutional layers with filters from 16 to 128. Receiver operating characteristic analysis was performed to evaluate prediction performance. For comparison, an expert radiologist also scored the same nodes as normal or abnormal.

RESULTS

The convolutional neural network model yielded a specificity of 79.3% ± 5.1%, sensitivity of 92.1% ± 2.9%, positive predictive value of 76.9% ± 4.0%, negative predictive value of 93.3% ± 1.9%, accuracy of 84.8% ± 2.4%, and receiver operating characteristic area under the curve of 0.91 ± 0.02 for the validation data set. These results compared favorably with scoring by radiologists (accuracy of 78%).

CONCLUSION

The results are encouraging and suggest that this approach may prove useful for classifying lymph node status on MRI in clinical settings in patients with breast cancer, although additional studies are needed before routine clinical use can be realized. This approach has the potential to ultimately be a noninvasive alternative to lymph node biopsy.

Machine learning in breast MRI

Machine-learning techniques have led to remarkable advances in data extraction and analysis of medical imaging. Applications of machine learning to breast MRI continue to expand rapidly as increasingly accurate 3D breast and lesion segmentation allows the combination of radiologist-level interpretation (eg, BI-RADS lexicon), data from advanced multiparametric imaging techniques, and patient-level data such as genetic risk markers. Advances in breast MRI feature extraction have led to rapid dataset analysis, which offers promise in large pooled multiinstitutional data analysis. The object of this review is to provide an overview of machine-learning and deep-learning techniques for breast MRI, including supervised and unsupervised methods, anatomic breast segmentation, and lesion segmentation. Finally, it explores the role of machine learning, current limitations, and future applications to texture analysis, radiomics, and radiogenomics. Level of Evidence: 3 Technical Efficacy Stage: 2 J. Magn. Reson. Imaging 2019. J. Magn. Reson. Imaging 2020;52:998-1018.

Artificial intelligence in breast imaging

This article reviews current limitations and future opportunities for the application of computer-aided detection (CAD) systems and artificial intelligence in breast imaging. Traditional CAD systems in mammography screening have followed a rules-based approach, incorporating domain knowledge into hand-crafted features before using classical machine learning techniques as a classifier. The first commercial CAD system, ImageChecker M1000, relies on computer vision techniques for pattern recognition. Unfortunately, CAD systems have been shown to adversely affect some radiologists' performance and increase recall rates. The Digital Mammography DREAM Challenge was a multidisciplinary collaboration that provided 640,000 mammography images for teams to help decrease false-positive rates in breast cancer screening. Winning solutions leveraged deep learning's (DL) automatic hierarchical feature learning capabilities and used convolutional neural networks. Start-ups Therapixel and Kheiron Medical Technologies are using DL for breast cancer screening. With increasing use of digital breast tomosynthesis, specific artificial intelligence (AI)-CAD systems are emerging to include iCAD's PowerLook Tomo Detection and ScreenPoint Medical's Transpara. Other AI-CAD systems are focusing on breast diagnostic techniques such as ultrasound and magnetic resonance imaging (MRI). There is a gap in the market for contrast-enhanced spectral mammography AI-CAD tools. Clinical implementation of AI-CAD tools requires testing in scenarios mimicking real life to prove its usefulness in the clinical environment. This requires a large and representative dataset for testing and assessment of the reader's interaction with the tools. A cost-effectiveness assessment should be undertaken, with a large feasibility study carried out to ensure there are no unintended consequences. AI-CAD systems should incorporate explainable AI in accordance with the European Union General Data Protection Regulation (GDPR).

Optimization of a Deep-Learning Method Based on the Classification of Images Generated by Parameterized Deep Snap a Novel Molecular-Image-Input Technique for Quantitative Structure-Activity Relationship (QSAR) Analysis

Numerous chemical compounds are distributed around the world and may affect the homeostasis of the endocrine system by disrupting the normal functions of hormone receptors. Although the risks associated with these compounds have been evaluated by acute toxicity testing in mammalian models, the chronic toxicity of many chemicals remains due to high cost of the compounds and the testing, etc. However, computational approaches may be promising alternatives and reduce these evaluations. Recently, deep learning (DL) has been shown to be promising prediction models with high accuracy for recognition of images, speech, signals, and videos since it greatly benefits from large datasets. Recently, a novel DL-based technique called DeepSnap was developed to conduct QSAR analysis using three-dimensional images of chemical structures. It can be used to predict the potential toxicity of many different chemicals to various receptors without extraction of descriptors. DeepSnap has been shown to have a very high capacity in tests using Tox21 quantitative qHTP datasets. Numerous parameters must be adjusted to use the DeepSnap method but they have not been optimized. In this study, the effects of these parameters on the performance of the DL prediction model were evaluated in terms of the loss in validation as an indicator for evaluating the performance of the DL using the toxicity information in the Tox21 qHTP database. The relations of the parameters of DeepSnap such as (1) number of molecules per SDF split into (2) zoom factor percentage, (3) atom size for van der waals percentage, (4) bond radius, (5) minimum bond distance, and (6) bond tolerance, with the validation loss following quadratic function curves, which suggests that optimal thresholds exist to attain the best performance with these prediction models. Using the parameter values set with the best performance, the prediction model of chemical compounds for CAR agonist was built using 64 images, at 105° angle, with AUC of 0.791. Thus, based on these parameters, the proposed DeepSnap-DL approach will be highly reliable and beneficial to establish models to assess the risk associated with various chemicals.