Comparison of CT and MRI images for the prediction of soft-tissue sarcoma grading and lung metastasis via a convolutional neural networks model

Diagnostic Value of Magnetic Resonance Imaging Radiomics and Machine-learning in Grading Soft Tissue Sarcoma: A Mini-review on the Current State

Soft tissue sarcomas (STS) are a heterogeneous group of rare malignant tumors. Tumor grade might be underestimated in biopsy due to intratumoral heterogeneity. This mini-review aims to present the current state of predicting malignancy grades of STS through radiomics, machine learning, and deep learning on magnetic resonance imaging (MRI). Several studies investigated various machine-learning and deep-learning approaches in T2-weighted (w) images, contrast-enhanced (CE) T1w images, and DWI/ADC maps with promising results. Combining semantic imaging features, radiomics features, and deep-learning signatures in machine-learning models has demonstrated superior predictive performances compared to individual feature sources. Furthermore, incorporating features from both tumor volume and peritumor region is beneficial. Especially random forest and support vector machine classifiers, often combined with the least absolute shrinkage and selection operator (LASSO) and/or synthetic minority oversampling technique (SMOTE), did show high area under the curve (AUC) values and accuracies in existing studies.

Radiomics of multi-parametric MRI for the prediction of lung metastasis in soft-tissue sarcoma: a feasibility study

PURPOSE

To investigate the value of multi-parametric MRI-based radiomics for preoperative prediction of lung metastases from soft tissue sarcoma (STS).

METHODS

In total, 122 patients with clinicopathologically confirmed STS who underwent pretreatment T1-weighted contrast-enhanced (T1-CE) and T2-weighted fat-suppressed (T2FS) MRI scans were enrolled between Jul. 2017 and Mar. 2021. Radiomics signatures were established by calculating and selecting radiomics features from the two sequences. Clinical independent predictors were evaluated by statistical analysis. The radiomics nomogram was constructed from margin and radiomics features by multivariable logistic regression. Finally, the study used receiver operating characteristic (ROC) and calibration curves to evaluate performance of radiomics models. Decision curve analyses (DCA) were performed to evaluate clinical usefulness of the models.

RESULTS

The margin was considered as an independent predictor (p < 0.05). A total of 4 MRI features were selected and used to develop the radiomics signature. By incorporating the margin and radiomics signature, the developed nomogram showed the best prediction performance in the training (AUCs, margin vs. radiomics signature vs. nomogram, 0.609 vs. 0.909 vs. 0.910) and validation (AUCs, margin vs. radiomics signature vs. nomogram, 0.666 vs. 0.841 vs. 0.894) cohorts. DCA indicated potential usefulness of the nomogram model.

CONCLUSIONS

This feasibility study evaluated predictive values of multi-parametric MRI for the prediction of lung metastasis, and proposed a nomogram model to potentially facilitate the individualized treatment decision-making for STSs.

A multimodal neural network with gradient blending improves predictions of survival and metastasis in sarcoma

The objective of this study is to develop a multimodal neural network (MMNN) model that analyzes clinical variables and MRI images of a soft tissue sarcoma (STS) patient, to predict overall survival and risk of distant metastases. We compare the performance of this MMNN to models based on clinical variables alone, radiomics models, and an unimodal neural network. We include patients aged 18 or older with biopsy-proven STS who underwent primary resection between January 1st, 2005, and December 31st, 2020 with complete outcome data and a pre-treatment MRI with both a T1 post-contrast sequence and a T2 fat-sat sequence available. A total of 9380 MRI slices containing sarcomas from 287 patients are available. Our MMNN accepts the entire 3D sarcoma volume from T1 and T2 MRIs and clinical variables. Gradient blending allows the clinical and image sub-networks to optimally converge without overfitting. Heat maps were generated to visualize the salient image features. Our MMNN outperformed all other models in predicting overall survival and the risk of distant metastases. The C-Index of our MMNN for overall survival is 0.77 and the C-Index for risk of distant metastases is 0.70. The provided heat maps demonstrate areas of sarcomas deemed most salient for predictions. Our multimodal neural network with gradient blending improves predictions of overall survival and risk of distant metastases in patients with soft tissue sarcoma. Future work enabling accurate subtype-specific predictions will likely utilize similar end-to-end multimodal neural network architecture and require prospective curation of high-quality data, the inclusion of genomic data, and the involvement of multiple centers through federated learning.

Inferring malignancy grade of soft tissue sarcomas from magnetic resonance imaging features: A systematic review

PURPOSE

Systematic reviews on the grading of STS using MRI are lacking. This review analyses the role of different MRI features in inferring the histological grade of STS.

MATERIALS AND METHODS

A systematic review was conducted and is reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) checklist. The electronic databases of PubMed/MEDLINE were systematically searched for literature addressing the correlation of MRI findings in soft tissue sarcoma with tumor grade. As keywords "MRI", "magnetic resonance imaging", "sarcoma", "grade", "grading", and "FNCLCC" have been selected.

RESULTS

14 studies have been included in this systematic review. Tumor size (p = 0.015 (51 patients) to p = 0.81 (36 patients)), tumor margin (p < 0.001 (95 patients) to 0.93 (36 patients)), necrosis (p = 0.004 (50 patients) to p = 0.65 (95 patients)), peritumoral edema (p = 0.002 (130 patients) to p = 0.337 (40 patients)), contrast enhancement (p < 0.01 (50 patients) to 0.019 (51 patients)) and polycyclic/multilobulated tumor configuration (p = 0.008 (71 patients)) were significantly associated with STS malignancy grade in most of the included studies. Heterogeneity in T2w images (p = 0.003 (130 patients) to 0.202 (40 patients)), signal intensity in T1w images/ hemorrhage (p = 0.02 (130 patients) to 0.5 (31 patients)), peritumoral contrast enhancement (p < 0.001 (95 patients) to 0.253 (51 patients)) and tumoral diffusion restriction (p = 0.01 (51 patients) to 0.53 (52 patients)) were regarded as significantly associated with FNCLCC grade in some of the studies which investigated these features. Most other MRI features were not significant.

CONCLUSION

Several MRI features, such as tumor size, necrosis, peritumoral edema, peritumoral contrast enhancement, intratumoral contrast enhancement, and polycyclic/multilobulated tumor configuration may indicate the malignancy grade of STS. However, further studies are needed to gain consensus.

Automated Prediction of Osteoarthritis Level in Human Osteochondral Tissue Using Histopathological Images

Osteoarthritis (OA) is the most common arthritis and the leading cause of lower extremity disability in older adults. Understanding OA progression is important in the development of patient-specific therapeutic techniques at the early stage of OA rather than at the end stage. Histopathology scoring systems are usually used to evaluate OA progress and the mechanisms involved in the development of OA. This study aims to classify the histopathological images of cartilage specimens automatically, using artificial intelligence algorithms. Hematoxylin and eosin (HE)- and safranin O and fast green (SafO)-stained images of human cartilage specimens were divided into early, mild, moderate, and severe OA. Five pre-trained convolutional networks (DarkNet-19, MobileNet, ResNet-101, NasNet) were utilized to extract the twenty features from the last fully connected layers for both scenarios of SafO and HE. Principal component analysis (PCA) and ant lion optimization (ALO) were utilized to obtain the best-weighted features. The support vector machine classifier was trained and tested based on the selected descriptors to achieve the highest accuracies of 98.04% and 97.03% in HE and SafO, respectively. Using the ALO algorithm, the F1 scores were 0.97, 0.991, 1, and 1 for the HE images and 1, 0.991, 0.97, and 1 for the SafO images for the early, mild, moderate, and severe classes, respectively. This algorithm may be a useful tool for researchers to evaluate the histopathological images of OA without the need for experts in histopathology scoring systems or the need to train new experts. Incorporating automated deep features could help to improve the characterization and understanding of OA progression and development.

Integrating features from lymph node stations for metastatic lymph node detection

Metastasis on lymph nodes (LNs), the most common way of spread for primary tumor cells, is a sign of increased mortality. However, metastatic LNs are time-consuming and challenging to detect even for professional radiologists due to their small sizes, high sparsity, and ambiguity in appearance. It is desired to leverage recent development in deep learning to automatically detect metastatic LNs. Besides a two-stage detection network, we here introduce an additional branch to leverage information about LN stations, an important reference for radiologists during metastatic LN diagnosis, as supplementary information for metastatic LN detection. The branch targets to solve a closely related task on the LN station level, i.e., classifying whether an LN station contains metastatic LN or not, so as to learn representations for LN stations. Considering that a metastatic LN station is expected to significantly affect the nearby ones, a GCN-based structure is adopted by the branch to model the relationship among different LN stations. At the classification stage of metastatic LN detection, the above learned LN station features, as well as the features reflecting the distance between the LN candidate and the LN stations, are integrated with the LN features. We validate our method on a dataset containing 114 intravenous contrast-enhanced Computed Tomography (CT) images of oral squamous cell carcinoma (OSCC) patients and show that it outperforms several state-of-the-art methods on the mFROC, maxF1, and AUC scores, respectively.

Applications of machine learning for imaging-driven diagnosis of musculoskeletal malignancies—a scoping review

Abstract

Musculoskeletal malignancies are a rare type of cancer. Consequently, sufficient imaging data for machine learning (ML) applications is difficult to obtain. The main purpose of this review was to investigate whether ML is already having an impact on imaging-driven diagnosis of musculoskeletal malignancies and what the respective reasons for this might be. A scoping review was conducted by a radiologist, an orthopaedic surgeon and a data scientist to identify suitable articles based on the PRISMA statement. Studies meeting the following criteria were included: primary malignant musculoskeletal tumours, machine/deep learning application, imaging data or data retrieved from images, human/preclinical, English language and original research. Initially, 480 articles were found and 38 met the eligibility criteria. Several continuous and discrete parameters related to publication, patient distribution, tumour specificities, ML methods, data and metrics were extracted from the final articles. For the synthesis, diagnosis-oriented studies were further examined by retrieving the number of patients and labels and metric scores. No significant correlations between metrics and mean number of samples were found. Several studies presented that ML could support imaging-driven diagnosis of musculoskeletal malignancies in distinct cases. However, data quality and quantity must be increased to achieve clinically relevant results. Compared to the experience of an expert radiologist, the studies used small datasets and mostly included only one type of data. Key to critical advancement of ML models for rare diseases such as musculoskeletal malignancies is a systematic, structured data collection and the establishment of (inter)national networks to obtain substantial datasets in the future.

Key Points

• Machine learning does not yet significantly impact imaging-driven diagnosis for musculoskeletal malignancies compared to other disciplines such as lung, breast or CNS cancer.

• Research in the area of musculoskeletal tumour imaging and machine learning is still very limited.

• Machine learning in musculoskeletal tumour imaging is impeded by insufficient availability of data and rarity of the disease.

Brain Tumor Classification Using a Combination of Variational Autoencoders and Generative Adversarial Networks

Brain tumors are a pernicious cancer with one of the lowest five-year survival rates. Neurologists often use magnetic resonance imaging (MRI) to diagnose the type of brain tumor. Automated computer-assisted tools can help them speed up the diagnosis process and reduce the burden on the health care systems. Recent advances in deep learning for medical imaging have shown remarkable results, especially in the automatic and instant diagnosis of various cancers. However, we need a large amount of data (images) to train the deep learning models in order to obtain good results. Large public datasets are rare in medicine. This paper proposes a framework based on unsupervised deep generative neural networks to solve this limitation. We combine two generative models in the proposed framework: variational autoencoders (VAEs) and generative adversarial networks (GANs). We swap the encoder-decoder network after initially training it on the training set of available MR images. The output of this swapped network is a noise vector that has information of the image manifold, and the cascaded generative adversarial network samples the input from this informative noise vector instead of random Gaussian noise. The proposed method helps the GAN to avoid mode collapse and generate realistic-looking brain tumor magnetic resonance images. These artificially generated images could solve the limitation of small medical datasets up to a reasonable extent and help the deep learning models perform acceptably. We used the ResNet50 as a classifier, and the artificially generated brain tumor images are used to augment the real and available images during the classifier training. We compared the classification results with several existing studies and state-of-the-art machine learning models. Our proposed methodology noticeably achieved better results. By using brain tumor images generated artificially by our proposed method, the classification average accuracy improved from 72.63% to 96.25%. For the most severe class of brain tumor, glioma, we achieved 0.769, 0.837, 0.833, and 0.80 values for recall, specificity, precision, and F1-score, respectively. The proposed generative model framework could be used to generate medical images in any domain, including PET (positron emission tomography) and MRI scans of various parts of the body, and the results show that it could be a useful clinical tool for medical experts.

Existing and Emerging Breast Cancer Detection Technologies and Its Challenges: A Review

Breast cancer is the most leading cancer occurring in women and is a significant factor in female mortality. Early diagnosis of breast cancer with Artificial Intelligent (AI) developments for breast cancer detection can lead to a proper treatment to affected patients as early as possible that eventually help reduce the women mortality rate. Reliability issues limit the current clinical detection techniques, such as Ultra-Sound, Mammography, and Magnetic Resonance Imaging (MRI) from screening images for precise elucidation. The capability to detect a tumor in early diagnosis, expensive, relatively long waiting time due to pandemic and painful procedure for a patient to perform. This article aims to review breast cancer screening methods and recent technological advancements systematically. In addition, this paper intends to explore the progression and challenges of AI in breast cancer detection. The next state of the art between image and signal processing will be presented, and their performance is compared. This review will facilitate the researcher to insight the view of breast cancer detection technologies advancement and its challenges.

A Highly Reliable Convolutional Neural Network Based Soft Tissue Sarcoma Metastasis Detection from Chest X-ray Images: A Retrospective Cohort Study

Simple Summary

Soft tissue sarcomas are relatively rare malignant diseases. Part of the diagnosis and follow-up includes medical imaging of the thorax for detection of lung metastases. A Python script was created and trained using a set of lung X-rays and concordant CT scans from a high-volume German-speaking sarcoma center. It is capable of detecting malignant metastasis in the lung with a precision of 71.2%, specificity of 90.5%, sensitivity of 94% and accuracy of 91.2%. Furthermore, the program was able to detect even small nodules with a size <1 cm in conventional X-rays of the thorax. This algorithm was implemented into our daily clinical practice alongside with the radiologists’ findings. With this tool we aim to improve the quality of our service and reduce the expenditure of time.

Abstract

Introduction: soft tissue sarcomas are a subset of malignant tumors that are relatively rare and make up 1% of all malignant tumors in adulthood. Due to the rarity of these tumors, there are significant differences in quality in the diagnosis and treatment of these tumors. One paramount aspect is the diagnosis of hematogenous metastases in the lungs. Guidelines recommend routine lung imaging by means of X-rays. With the ever advancing AI-based diagnostic support, there has so far been no implementation for sarcomas. The aim of the study was to utilize AI to obtain analyzes regarding metastasis on lung X-rays in the most possible sensitive and specific manner in sarcoma patients. Methods: a Python script was created and trained using a set of lung X-rays with sarcoma metastases from a high-volume German-speaking sarcoma center. 26 patients with lung metastasis were included. For all patients chest X-ray with corresponding lung CT scans, and histological biopsies were available. The number of trainable images were expanded to 600. In order to evaluate the biological sensitivity and specificity, the script was tested on lung X-rays with a lung CT as control. Results: in this study we present a new type of convolutional neural network-based system with a precision of 71.2%, specificity of 90.5%, sensitivity of 94%, recall of 94% and accuracy of 91.2%. A good detection of even small findings was determined. Discussion: the created script establishes the option to check lung X-rays for metastases at a safe level, especially given this rare tumor entity.

Imaging response evaluation after neoadjuvant treatment in soft tissue sarcomas: Where do we stand?

Soft tissue sarcomas (STS) represent a broad family of rare tumours for which surgery with radiotherapy represents first-line treatment. Recently, neoadjuvant chemo-radiotherapy has been increasingly used in high-risk patients in an effort to reduce surgical morbidity and improve clinical outcomes. An adequate understanding of the efficacy of neoadjuvant therapies would optimise patient care, allowing a tailored approach. Although response evaluation criteria in solid tumours (RECIST) is the most common imaging method to assess tumour response, Choi criteria and functional and molecular imaging (DWI, DCE-MRI and ¹⁸F-FDG-PET) seem to outperform it in the discrimination between responders and non-responders. Moreover, the radiologic-pathology correlation of treatment-related changes remains poorly understood. In this review, we provide an overview of the imaging assessment of tumour response in STS undergoing neoadjuvant treatment, including conventional imaging (CT, MRI, PET) and advanced imaging analysis. Future directions will be presented to shed light on potential advances in pre-surgical imaging assessments that have clinical implications for sarcoma patients.

Use of deep learning for detection, characterisation and prediction of metastatic disease from computerised tomography: a systematic review

CT is widely used for diagnosis, staging and management of cancer. The presence of metastasis has significant implications on treatment and prognosis. Deep learning (DL), a form of machine learning, where layers of programmed algorithms interpret and recognise patterns, may have a potential role in CT image analysis. This review aims to provide an overview on the use of DL in CT image analysis in the diagnostic evaluation of metastatic disease. A total of 29 studies were included which could be grouped together into three areas of research: the use of deep learning on the detection of metastatic disease from CT imaging, characterisation of lesions on CT into metastasis and prediction of the presence or development of metastasis based on the primary tumour. In conclusion, DL in CT image analysis could have a potential role in evaluating metastatic disease; however, prospective clinical trials investigating its clinical value are required.

Artificial intelligence performance in detecting tumor metastasis from medical radiology imaging: A systematic review and meta-analysis

BACKGROUND

Early diagnosis of tumor metastasis is crucial for clinical treatment. Artificial intelligence (AI) has shown great promise in the field of medicine. We therefore aimed to evaluate the diagnostic accuracy of AI algorithms in detecting tumor metastasis using medical radiology imaging.

METHODS

We searched PubMed and Web of Science for studies published from January 1, 1997, to January 30, 2020. Studies evaluating an AI model for the diagnosis of tumor metastasis from medical images were included. We excluded studies that used histopathology images or medical wave-form data and those focused on the region segmentation of interest. Studies providing enough information to construct contingency tables were included in a meta-analysis.

FINDINGS

We identified 2620 studies, of which 69 were included. Among them, 34 studies were included in a meta-analysis with a pooled sensitivity of 82% (95% CI 79-84%), specificity of 84% (82-87%) and AUC of 0·90 (0·87-0·92). Analysis for different AI algorithms showed a pooled sensitivity of 87% (83-90%) for machine learning and 86% (82-89%) for deep learning, and a pooled specificity of 89% (82-93%) for machine learning, and 87% (82-91%) for deep learning.

INTERPRETATION

AI algorithms may be used for the diagnosis of tumor metastasis using medical radiology imaging with equivalent or even better performance to health-care professionals, in terms of sensitivity and specificity. At the same time, rigorous reporting standards with external validation and comparison to health-care professionals are urgently needed for AI application in the medical field.

FUNDING

College students' innovative entrepreneurial training plan program .

Differentiation of COVID-19 conditions in planar chest radiographs using optimized convolutional neural networks

In this study, an attempt has been made to differentiate Novel Coronavirus-2019 (COVID-19) conditions from healthy subjects in Chest radiographs using a simplified end-to-end Convolutional Neural Network (CNN) model and occlusion sensitivity maps. Early detection and faster automated screening of the COVID-19 patients is essential. For this, the images are considered from publicly available datasets. Significant biomarkers representing critical image features are extracted from CNN by experimentally investigating on cross-validation methods and hyperparameter settings. The performance of the network is evaluated using standard metrics. Perturbation based occlusion sensitivity maps are employed on the features obtained from the classification model to visualise the localization of abnormal areas. Results demonstrate that the simplified CNN model with optimised parameters is able to extract significant features with a sensitivity of 97.35% and F-measure of 96.71% to detect COVID-19 images. The algorithm achieves an Area Under the Curve-Receiver Operating Characteristic score of 99.4% with Matthews correlation coefficient of 0.93. High value of Diagnostic odds ratio is also obtained. Occlusion sensitivity maps provide precise localization of abnormal regions by identifying COVID-19 conditions. As early detection through chest radiographic images are useful for automated screening of the disease, this method appears to be clinically relevant in providing a visual diagnostic solution using a simplified and efficient model.

Modelling the skeletal muscle injury recovery using in vivo contrast-enhanced micro-CT: a proof-of-concept study in a rat model

Background

Skeletal muscle injury characterisation during healing supports trauma prognosis. Given the potential interest of computed tomography (CT) in muscle diseases and lack of in vivo CT methodology to image skeletal muscle wound healing, we tracked skeletal muscle injury recovery using in vivo micro-CT in a rat model to obtain a predictive model.

Methods

Skeletal muscle injury was performed in 23 rats. Twenty animals were sorted into five groups to image lesion recovery at 2, 4, 7, 10, or 14 days after injury using contrast-enhanced micro-CT. Injury volumes were quantified using a semiautomatic image processing, and these values were used to build a prediction model. The remaining 3 rats were imaged at all monitoring time points as validation. Predictions were compared with Bland-Altman analysis.

Results

Optimal contrast agent dose was found to be 20 mL/kg injected at 400 μL/min. Injury volumes showed a decreasing tendency from day 0 (32.3 ± 12.0mm³, mean ± standard deviation) to day 2, 4, 7, 10, and 14 after injury (19.6 ± 12.6, 11.0 ± 6.7, 8.2 ± 7.7, 5.7 ± 3.9, and 4.5 ± 4.8 mm³, respectively). Groups with single monitoring time point did not yield significant differences with the validation group lesions. Further exponential model training with single follow-up data (R² = 0.968) to predict injury recovery in the validation cohort gave a predictions root mean squared error of 6.8 ± 5.4 mm³. Further prediction analysis yielded a bias of 2.327.

Conclusion

Contrast-enhanced CT allowed in vivo tracking of skeletal muscle injury recovery in rat.