Computer-aided diagnosis of soft-tissue tumors using sonographic morphologic and texture features

Deep learning driven diagnosis of malignant soft tissue tumors based on dual-modal ultrasound images and clinical indexes

Background

Soft tissue tumors (STTs) are benign or malignant superficial neoplasms arising from soft tissues throughout the body with versatile pathological types. Although Ultrasonography (US) is one of the most common imaging tools to diagnose malignant STTs, it still has several drawbacks in STT diagnosis that need improving.

Objectives

The study aims to establish this deep learning (DL) driven Artificial intelligence (AI) system for predicting malignant STTs based on US images and clinical indexes of the patients.

Methods

We retrospectively enrolled 271 malignant and 462 benign masses to build the AI system using 5-fold validation. A prospective dataset of 44 malignant masses and 101 benign masses was used to validate the accuracy of system. A multi-data fusion convolutional neural network, named ultrasound clinical soft tissue tumor net (UC-STTNet), was developed to combine gray scale and color Doppler US images and clinic features for malignant STTs diagnosis. Six radiologists (R1-R6) with three experience levels were invited for reader study.

Results

The AI system achieved an area under receiver operating curve (AUC) value of 0.89 in the retrospective dataset. The diagnostic performance of the AI system was higher than that of one of the senior radiologists (AUC of AI vs R2: 0.89 vs. 0.84, p=0.022) and all of the intermediate and junior radiologists (AUC of AI vs R3, R4, R5, R6: 0.89 vs 0.75, 0.81, 0.80, 0.63; p <0.01). The AI system also achieved an AUC of 0.85 in the prospective dataset. With the assistance of the system, the diagnostic performances and inter-observer agreement of the radiologists was improved (AUC of R3, R5, R6: 0.75 to 0.83, 0.80 to 0.85, 0.63 to 0.69; p<0.01).

Conclusion

The AI system could be a useful tool in diagnosing malignant STTs, and could also help radiologists improve diagnostic performance.

Deep Learning-Based Evaluation of Ultrasound Images for Benign Skin Tumors

In this study, a combined convolutional neural network for the diagnosis of three benign skin tumors was designed, and its effectiveness was verified through quantitative and statistical analysis. To this end, 698 sonographic images were taken and diagnosed at the Department of Dermatology at Severance Hospital in Seoul, Korea, between 10 November 2017 and 17 January 2020. Through an empirical process, a convolutional neural network combining two structures, which consist of a residual structure and an attention-gated structure, was designed. Five-fold cross-validation was applied, and the train set for each fold was augmented by the Fast AutoAugment technique. As a result of training, for three benign skin tumors, an average accuracy of 95.87%, an average sensitivity of 90.10%, and an average specificity of 96.23% were derived. Also, through statistical analysis using a class activation map and physicians' findings, it was found that the judgment criteria of physicians and the trained combined convolutional neural network were similar. This study suggests that the model designed and trained in this study can be a diagnostic aid to assist physicians and enable more efficient and accurate diagnoses.

Application of artificial intelligence to imaging interpretations in the musculoskeletal area: Where are we? Where are we going?

The interest of researchers, clinicians and radiologists, in artificial intelligence (AI) continues to grow. Deep learning is a subset of machine learning, in which the computer algorithm itself can determine the optimal imaging features to answer a clinical question. Convolutional neural networks are the most common architecture for performing deep learning on medical images. The various musculoskeletal applications of deep learning are the detection of abnormalities on X-rays or cross-sectional images (CT, MRI), for example the detection of fractures, meniscal tears, anterior cruciate ligament tears, degenerative lesions of the spine, bone metastases, classification of e.g., dural sac stenosis, degeneration of intervertebral discs, assessment of skeletal age, and segmentation, for example of cartilage. Software developments are already impacting the daily practice of orthopedic imaging by automatically detecting fractures on radiographs. Improving image acquisition protocols, improving the quality of low-dose CT images, reducing acquisition times in MRI, or improving MR image resolution is possible through deep learning. Deep learning offers an automated way to offload time-consuming manual processes and improve practitioner performance. This article reviews the current state of AI in musculoskeletal imaging.

AI in Breast Cancer Imaging: A Survey of Different Applications

Breast cancer was the most diagnosed cancer in 2020. Several thousand women continue to die from this disease. A better and earlier diagnosis may be of great importance to improving prognosis, and that is where Artificial Intelligence (AI) could play a major role. This paper surveys different applications of AI in Breast Imaging. First, traditional Machine Learning and Deep Learning methods that can detect the presence of a lesion and classify it into benign/malignant-which could be important to diminish reading time and improve accuracy-are analyzed. Following that, researches in the field of breast cancer risk prediction using mammograms-which may be able to allow screening programs customization both on periodicity and modality-are reviewed. The subsequent section analyzes different applications of augmentation techniques that allow to surpass the lack of labeled data. Finally, still concerning the absence of big datasets with labeled data, the last section studies Self-Supervised learning, where AI models are able to learn a representation of the input by themselves. This review gives a general view of what AI can give in the field of Breast Imaging, discussing not only its potential but also the challenges that still have to be overcome.

Artificial intelligence applied to musculoskeletal oncology: a systematic review

Developments in artificial intelligence have the potential to improve the care of patients with musculoskeletal tumors. We performed a systematic review of the published scientific literature to identify the current state of the art of artificial intelligence applied to musculoskeletal oncology, including both primary and metastatic tumors, and across the radiology, nuclear medicine, pathology, clinical research, and molecular biology literature. Through this search, we identified 252 primary research articles, of which 58 used deep learning and 194 used other machine learning techniques. Articles involving deep learning have mostly involved bone scintigraphy, histopathology, and radiologic imaging. Articles involving other machine learning techniques have mostly involved transcriptomic analyses, radiomics, and clinical outcome prediction models using medical records. These articles predominantly present proof-of-concept work, other than the automated bone scan index for bone metastasis quantification, which has translated to clinical workflows in some regions. We systematically review and discuss this literature, highlight opportunities for multidisciplinary collaboration, and identify potentially clinically useful topics with a relative paucity of research attention. Musculoskeletal oncology is an inherently multidisciplinary field, and future research will need to integrate and synthesize noisy siloed data from across clinical, imaging, and molecular datasets. Building the data infrastructure for collaboration will help to accelerate progress towards making artificial intelligence truly useful in musculoskeletal oncology.

Systematic review of sarcomas radiomics studies: Bridging the gap between concepts and clinical applications?

OBJECTIVES

Sarcomas are a model for intra- and inter-tumoral heterogeneities making them particularly suitable for radiomics analyses. Our purposes were to review the aims, methods and results of radiomics studies involving sarcomas METHODS: Pubmed and Web of Sciences databases were searched for radiomics or textural studies involving bone, soft-tissues and visceral sarcomas until June 2020. Two radiologists evaluated their objectives, results and quality of their methods, imaging pre-processing and machine-learning workflow helped by the items of the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2), Image Biomarker Standardization Initiative (IBSI) and 'Radiomics Quality Score' (RQS). Statistical analyses included inter-reader agreements, correlations between methodological assessments, scientometrics indices, and their changes over years, and between RQS, number of patients and models performance.

RESULTS

Fifty-two studies were included involving: soft-tissue sarcomas (29/52, 55.8 %), bone sarcomas (15/52, 28.8 %), gynecological sarcomas (6/52, 11.5 %) and mixed sarcomas (2/52, 3.8 %), mostly imaged with MRI (36/52, 69.2 %), for a total of distinct patients. Median RQS was 4.5 (28.4 % of the maximum, range: -7 - 17). Performances of predictive models and number of patients negatively correlated (p = 0.027). None of the studies detailed all the items from the IBSI guidelines. There was a significant increase in studies' impact factors since the establishing of the RQS in 2017 (p = 0.038).

CONCLUSION

Although showing promising results, further efforts are needed to make sarcoma radiomics studies reproducible with an acceptable level of evidence. A better knowledge of the RQS and IBSI reporting guidelines could improve the quality of sarcoma radiomics studies and accelerate clinical applications.

Applications of Artificial Intelligence in Musculoskeletal Imaging: From the Request to the Report

Artificial intelligence (AI) will transform every step in the imaging value chain, including interpretive and noninterpretive components. Radiologists should familiarize themselves with AI developments to become leaders in their clinical implementation. This article explores the impact of AI through the entire imaging cycle of musculoskeletal radiology, from the placement of the requisition to the generation of the report, with an added Canadian perspective. Noninterpretive tasks which may be assisted by AI include the ordering of appropriate imaging tests, automatic exam protocoling, optimized scheduling, shorter magnetic resonance imaging acquisition time, computed tomography imaging with reduced artifact and radiation dose, and new methods of generation and utilization of radiology reports. Applications of AI for image interpretation consist of the determination of bone age, body composition measurements, screening for osteoporosis, identification of fractures, evaluation of segmental spine pathology, detection and temporal monitoring of osseous metastases, diagnosis of primary bone and soft tissue tumors, and grading of osteoarthritis.

Breast tumor classification using different features of quantitative ultrasound parametric images

RATIONALE AND OBJECTIVES

The ultrasound B-mode-based morphological and texture analysis and Nakagami parametric imaging have been proposed to characterize breast tumors. Since these three feature categories of ultrasonic tissue characterization supply information on different physical characteristics of breast tumors, by combining the above methods is expected to provide more clues for classifying breast tumors.

MATERIALS AND METHODS

To verify the validity of the concept, raw data were obtained from 160 clinical cases. Six different types of morphological-feature parameters, four texture features, and the Nakagami parameter of benignancy and malignancy were extracted for evaluation. The Pearson's correlation matrix was used to calculate the correlation between different feature parameters. The fuzzy c-means clustering and stepwise regression techniques were utilized to determine the optimal feature set, respectively. The logistic regression, receiver operating characteristic curve, and support vector machine were used to estimate the diagnostic ability.

RESULTS

The best performance was obtained by combining morphological-feature parameter (e.g., standard deviation of the shortest distance), texture feature (e.g., variance), and the Nakagami parameter, with an accuracy of 89.4%, a specificity of 86.3%, a sensitivity of 92.5%, and an area under receiver operating characteristic curve of 0.96. There was no significant difference between using fuzzy c-means clustering, logistic regression, and support vector machine based on the optimal feature set for breast tumors classification.

CONCLUSION

Therefore, we verified that different physical ultrasonic features are functionally complementary and thus improve the performance in diagnosing breast tumors. Moreover, the optimal feature set had the maximum discriminating performance should be irrelative to the power of classifiers.

Preoperative Ultrasonographic Evaluation for Malignancy of Soft-Tissue Sarcoma: A Retrospective Study

Background:

Ultrasonography is useful for distinguishing between benign and malignant soft-tissue tumors. However, no study has focused on its usefulness in the differential diagnosis between low-grade and high-grade soft-tissue sarcomas. We conducted a retrospective study to determine the usefulness of the parameters of ultrasonograph and to develop a practical scoring system for distinguishing between high-grade and low-grade sarcomas.

Methods:

Twenty-two cases of low-grade and 43 cases of high-grade malignant soft-tissue sarcoma were enrolled. Ultrasonography parameters including the longest diameter, depth of the tumor, echogenicity, tumor margin, and vascularity defined according to Giovagnorio’s criteria were analyzed as factors to distinguish between the two types of sarcoma. Significant factors were entered into a multivariate model to define the scores for distinction according to the odds ratio. The usefulness of the score was analyzed via receiver operating characteristic analyses.

Results:

In univariate analysis, tumor margin, echogenicity, and vascularity were significantly different between low- and high-grade sarcomas. In the multivariate regression model, the odds ratio for high-grade vs. low-grade sarcoma was 8.8 for tumor margin, 69 for echogenicity, and 8.3 for vascularity. Scores for the risk factors were defined as follows: 1, ill-defined margin; 2, hypoechoic echogenicity; and 1, type IV in Giovagnorio’s criteria. The sum of each score was confirmed by receiver operating characteristic analysis. The area under the curve was 0.95, with a cut-off score of 3, indicating that the scoring system was useful.

Conclusion:

The ultrasonography parameters of tumor margin, echogenicity, and vascularity are useful for distinguishing between low- and high-grade sarcomas.

Imaging of Soft-Tissue Musculoskeletal Masses: Fundamental Concepts

Radiologic evaluation of musculoskeletal soft-tissue masses has changed dramatically with the continued improvements in imaging technology. The integration of advanced imaging has provided the radiologist with a wide range of assessment tools, but as with all powerful arsenals, selection and application of the appropriate imaging method can be problematic. Although the choices available for imaging evaluation of musculoskeletal masses have changed dramatically, the basic objectives of this assessment have remained constant: diagnosis and staging. The basic principles for evaluating musculoskeletal soft-tissue masses and achieving these objectives have not changed. This article addresses application of the current imaging methods to assessment of soft-tissue musculoskeletal masses, emphasizing fundamental concepts. We do not intend to provide a comprehensive review of imaging techniques, but rather to provide a useful review of the concepts needed to select the appropriate initial imaging method, magnetic resonance (MR) imaging field of view, MR imaging sequences, contrast material requirements, and rapid image acquisition techniques. We also address use of the new quantitative techniques of chemical shift and diffusion-weighted imaging. Finally, we review the current uses of computed tomography and ultrasonography. Although the choices available for imaging evaluation of musculoskeletal masses have changed dramatically within the past decade, appropriate application of the fundamental concepts of imaging will maximize the diagnostic utility of imaging examinations. ^©RSNA, 2016.

A Fusion-Based Approach for Breast Ultrasound Image Classification Using Multiple-ROI Texture and Morphological Analyses

Ultrasound imaging is commonly used for breast cancer diagnosis, but accurate interpretation of breast ultrasound (BUS) images is often challenging and operator-dependent. Computer-aided diagnosis (CAD) systems can be employed to provide the radiologists with a second opinion to improve the diagnosis accuracy. In this study, a new CAD system is developed to enable accurate BUS image classification. In particular, an improved texture analysis is introduced, in which the tumor is divided into a set of nonoverlapping regions of interest (ROIs). Each ROI is analyzed using gray-level cooccurrence matrix features and a support vector machine classifier to estimate its tumor class indicator. The tumor class indicators of all ROIs are combined using a voting mechanism to estimate the tumor class. In addition, morphological analysis is employed to classify the tumor. A probabilistic approach is used to fuse the classification results of the multiple-ROI texture analysis and morphological analysis. The proposed approach is applied to classify 110 BUS images that include 64 benign and 46 malignant tumors. The accuracy, specificity, and sensitivity obtained using the proposed approach are 98.2%, 98.4%, and 97.8%, respectively. These results demonstrate that the proposed approach can effectively be used to differentiate benign and malignant tumors.

Doppler ultrasound for diagnosis of soft tissue sarcoma: efficacy of ultrasound-based screening score

BACKGROUND

The utility of ultrasound imaging in the screening of soft-part tumours (SPTs) has been reported. We classified SPTs according to their blood flow pattern on Doppler ultrasound and re-evaluated the efficacy of this imaging modality as a screening method. Additionally, we combined Doppler ultrasound with several values to improve the diagnostic efficacy and to establish a new diagnostic tool.

PATIENTS AND METHODS

This study included 189 cases of pathologically confirmed SPTs (122 cases of benign disease including SPTs and tumour-like lesions and 67 cases of malignant SPTs). Ultrasound imaging included evaluation of vascularity by colour Doppler. We established a scoring system to more effectively differentiate malignant from benign SPTs (ultrasound-based sarcoma screening [USS] score).

RESULTS

The mean scores in the benign and malignant groups were 1.47 ± 0.93 and 3.42 ± 1.30, respectively. Patients with malignant masses showed significantly higher USS scores than did those with benign masses (p < 1 × 10(-10)). The area under the curve was 0.88 by receiver operating characteristic (ROC) analysis. Based on the cut-off value (3 points) calculated by ROC curve analysis, the sensitivity and specificity for a diagnosis of malignant SPT was 85.1% and 86.9%, respectively.

CONCLUSIONS

Assessment of vascularity by Doppler ultrasound alone is insufficient for differentiation between benign and malignant SPTs. Preoperative diagnosis of most SPTs is possible by combining our USS score with characteristic clinical and magnetic resonance imaging findings.

Differentiation of lipoma from liposarcoma on MRI using texture and shape analysis

RATIONALE AND OBJECTIVES

To determine if differentiation of lipoma from liposarcoma on magnetic resonance imaging can be improved using computer-assisted diagnosis (CAD).

MATERIALS AND METHODS

Forty-four histologically proven lipomatous tumors (24 lipomas and 20 liposarcomas) were studied retrospectively. Studies were performed at 1.5T and included T1-weighted, T2-weighted, T2-fat-suppressed, short inversion time inversion recovery, and contrast-enhanced sequences. Two experienced musculoskeletal radiologists blindly and independently noted their degree of confidence in malignancy using all available images/sequences for each patient. For CAD, tumors were segmented in three dimensions using T1-weighted images. Gray-level co-occurrence and run-length matrix textural features, as well as morphological features, were extracted from each tumor volume. Combinations of shape and textural features were used to train multiple, linear discriminant analysis classifiers. We assessed sensitivity, specificity, and accuracy of each classifier for delineating lipoma from liposarcoma using 10-fold cross-validation. Diagnostic accuracy of the two radiologists was determined using contingency tables. Interreader agreement was evaluated by Cohen kappa.

RESULTS

Using optimum-threshold criteria, CAD produced superior values (sensitivity, specificity, and accuracy are 85%, 96%, and 91%, respectively) compared to radiologist A (75%, 83%, and 80%) and radiologist B (80%, 75%, and 77%). Interreader agreement between radiologists was substantial (kappa [95% confidence interval]=0.69 [0.48-0.90]).

CONCLUSIONS

CAD may help radiologists distinguish lipoma from liposarcoma.

Quantitative analysis of colorectal lesions observed on magnified endoscopy images

BACKGROUND

Various surface mucosal pit patterns, as recognized by endoscopists, correlate with the histologic features of colorectal cancers. We investigated whether magnified endoscopy images of these pit patterns could be analyzed quantitatively and thus facilitate computer-aided diagnosis of colorectal lesions.

METHODS

We applied both texture analysis and scale-invariant feature transform (SIFT) descriptors and discriminant analysis to magnified endoscopy images of 165 neoplastic colorectal lesions (pit patterns: type III(L)/IV, n = 44; type V(I)-mildly irregular, n = 36; type V(I)-severely irregular, n = 45; type V(N), n = 40) [histologic findings: tubular adenoma (TA), n = 56; carcinoma with intramucosal or even scant submucosal invasion (M/SM-s), n = 52, carcinoma with massive submucosal invasion (SM-m), n = 57]. We analyzed differences in pit pattern values and corresponding histologic values to determine whether the values were diagnostically meaningful.

RESULTS

Gray-level difference matrix (GLDM) inverse difference moment and spatial gray-level dependence matrix (SGLDM) local homogeneity values differed significantly between type III(L)/IV and type V(N) pit patterns. Values differed significantly for each analyzed feature between type III(L)/IV and type V(I)-severely irregular patterns and were high but descending for type III(L)/IV, type V(I)-mildly irregular, and type V(I)-severely irregular pit patterns (in that order). Similarly, texture analysis yielded high but descending values for TA, M/SM-s, and SM-m (in that order). Furthermore, SIFT descriptors and discriminant analysis yielded differences that were superior to those obtained by texture analyses.

CONCLUSIONS

Computer analysis of magnified endoscopy images for the diagnosis of colorectal lesions appears feasible. We anticipate further developments in the computer-aided diagnosis of pit patterns on magnified endoscopy images.