A computer-aided diagnostic system to characterize CT focal liver lesions: design and optimization of a neural network classifier

Artificial intelligence for detection and characterization of focal hepatic lesions: a review

Focal liver lesions (FLL) are common incidental findings in abdominal imaging. While the majority of FLLs are benign and asymptomatic, some can be malignant or pre-malignant, and need accurate detection and classification. Current imaging techniques, such as computed tomography (CT) and magnetic resonance imaging (MRI), play a crucial role in assessing these lesions. Artificial intelligence (AI), particularly deep learning (DL), offers potential solutions by analyzing large data to identify patterns and extract clinical features that aid in the early detection and classification of FLLs. This manuscript reviews the diagnostic capacity of AI-based algorithms in processing CT and MRIs to detect benign and malignant FLLs, with an emphasis in the characterization and classification of these lesions and focusing on differentiating benign from pre-malignant and potentially malignant lesions. A comprehensive literature search from January 2010 to April 2024 identified 45 relevant studies. The majority of AI systems employed convolutional neural networks (CNNs), with expert radiologists providing reference standards through manual lesion delineation, and histology as the gold standard. The studies reviewed indicate that AI-based algorithms demonstrate high accuracy, sensitivity, specificity, and AUCs in detecting and characterizing FLLs. These algorithms excel in differentiating between benign and malignant lesions, optimizing diagnostic protocols, and reducing the needs of invasive procedures. Future research should concentrate on the expansion of data sets, the improvement of model explainability, and the validation of AI tools across a range of clinical setting to ensure the applicability and reliability of such tools.

A multi-modal deep neural network for multi-class liver cancer diagnosis

Liver disease is a potentially asymptomatic clinical entity that may progress to patient death. This study proposes a multi-modal deep neural network for multi-class malignant liver diagnosis. In parallel with the portal venous computed tomography (CT) scans, pathology data is utilized to prognosticate primary liver cancer variants and metastasis. The processed CT scans are fed to the deep dilated convolution neural network to explore salient features. The residual connections are further added to address vanishing gradient problems. Correspondingly, five pathological features are learned using a wide and deep network that gives a benefit of memorization with generalization. The down-scaled hierarchical features from CT scan and pathology data are concatenated to pass through fully connected layers for classification between liver cancer variants. In addition, the transfer learning of pre-trained deep dilated convolution layers assists in handling insufficient and imbalanced dataset issues. The fine-tuned network can predict three-class liver cancer variants with an average accuracy of 96.06% and an Area Under Curve (AUC) of 0.832. To the best of our knowledge, this is the first study to classify liver cancer variants by integrating pathology and image data, hence following the medical perspective of malignant liver diagnosis. The comparative analysis on the benchmark dataset shows that the proposed multi-modal neural network outperformed most of the liver diagnostic studies and is comparable to others.

Deep learning techniques in liver tumour diagnosis using CT and MR imaging - A systematic review

Deep learning has become a thriving force in the computer aided diagnosis of liver cancer, as it solves extremely complicated challenges with high accuracy over time and facilitates medical experts in their diagnostic and treatment procedures. This paper presents a comprehensive systematic review on deep learning techniques applied for various applications pertaining to liver images, challenges faced by the clinicians in liver tumour diagnosis and how deep learning bridges the gap between clinical practice and technological solutions with an in-depth summary of 113 articles. Since, deep learning is an emerging revolutionary technology, recent state-of-the-art research implemented on liver images are reviewed with more focus on classification, segmentation and clinical applications in the management of liver diseases. Additionally, similar review articles in literature are reviewed and compared. The review is concluded by presenting the contemporary trends and unaddressed research issues in the field of liver tumour diagnosis, offering directions for future research in this field.

РОЛЬ АЛГОРИТМОВ ИСКУССТВЕННОГО ИНТЕЛЛЕКТА В ДИАГНОСТИКЕ

Hepatosellülyar karsinoma (HSK) ən çox yayılan bədxassəli törəmələr arasında beşinci yeri tutur və dünyada xərçənglə əlaqəli ölümün üçüncü ən çox yayılmış səbəbidir. Süni intellekt (Sİ) sürətlə artan maraq sahəsidir. Müəlliflər HSK-ın diaqnostikasında və qiymətləndirilməsində Sİ-nin tətbiqi barədə məlumat verən məqalələri araşdırmışlar. Bu məqsədlə 27 məqalə təhlil edilmişdir. Təhlil edilmiş məqalələrdən KT görüntülərinin tədqiqinə dair 19 məqalədə (41,30%), USQ görüntülərinin öyrənilməsini əks etdirən 20 (43,47%) və MRT görüntülərindən bəhs edən 7 məqalədə (15,21%) müxtəlif Sİ alqoritmləri qəbul edilmişdir. Heç bir məqalədə PET və rentgen texnologiyasında süni intellektin istifadəsi müzakirə edilməyib. Sistematik yanaşma göstərmişdir ki, HSK-nin diaqnostikası və qiymətləndirilməsi üzrə əvvəlki işlərdə USQ, KT və MRT istifadə edilərək ənənəvi şərhin maşın öyrənməsi ilə müqayisəliliyi qiymətləndirilmişdir. Təhlillərimizdə görüntüləmə üsullarının istifadəsi HSK diaqnostikası üçün tibbi görüntüləmənin faydalılığını və təkamülünü əks etdirir. Bundan əlavə, nəticələrimiz lazımsız təkrarlanmanı və resursların israfını minimuma endirmək üçün birgə məlumat bazasında məlumat mübadiləsinə qaçılmaz ehtiyac olduğunu vurğulayır.

Гепатоцеллюлярная карцинома является пятым по распространенности злокачественным новообразованием и третьей по частоте причиной смерти от рака во всём мире. Искусственный интеллект — это быстрорастущая область интересов. Авторами были рассмотрены статьи, в которых сообщается о применении алгоритмов ИИ в диагностике и оценке ГЦК. Для этого проанализированы 27 статей. В проанализированных статьях в 19 статьях, посвящённых КТ-изображениям (41,30%), в 20 статьях, посвящённых изображениям УЗИ (43,47%), и в 7 статьях, посвящённым МРТ-изображениям (15,21%), использовали разные алгоритмы ИИ. Ни в одной статье не обсуждалось использование искусственного интеллекта в ПЭТ и рентгеновские технологии. Системный подход показал, что предыдущая работа по диагностике и оценке ГЦК оценивала сопоставимость традиционной интерпретации с машинным обучением с использованием УЗИ, КТ и МРТ. Использование методов визуализации в проведенном анализе отражает полезность и эволюцию медицинской визуализации для диагностики ГЦК. Кроме того, результаты поиска литературы подчёркивают острую необходимость совместного использования данных в совместных базах данных, чтобы свести к минимуму ненужное дублирование и растрату ресурсов.

Hepatocellular carcinoma (HCC) is the fifth most common malignancy and the third leading cause of cancer death worldwide. Artificial intelligence (AI) is a rapidly growing area of interest. We have reviewed articles reporting the application of AI algorithms in the diagnosis and evaluation of HCC. To do this, we analyzed 27 articles. In the analyzed articles, 19 articles on CT images (41.30%), 20 articles on ultrasound images (43.47%), and 7 articles on MRI images (15.21%) used different AI algorithms. None of the articles discussed the use of artificial intelligence in PET and X-ray technologies. Our systematic approach showed that previous work on the diagnosis and evaluation of HCC assessed the comparability of traditional interpretation with machine learning using ultrasound, CT, and MRI. The use of imaging modalities in our analysis reflects the usefulness and evolution of medical imaging for diagnosing HCC. In addition, our results highlight the critical need to share data across collaborative databases to minimize unnecessary duplication and waste of resources.

Semantic segmentation in medical images through transfused convolution and transformer networks

Recent decades have witnessed rapid development in the field of medical image segmentation. Deep learning-based fully convolution neural networks have played a significant role in the development of automated medical image segmentation models. Though immensely effective, such networks only take into account localized features and are unable to capitalize on the global context of medical image. In this paper, two deep learning based models have been proposed namely USegTransformer-P and USegTransformer-S. The proposed models capitalize upon local features and global features by amalgamating the transformer-based encoders and convolution-based encoders to segment medical images with high precision. Both the proposed models deliver promising results, performing better than the previous state of the art models in various segmentation tasks such as Brain tumor, Lung nodules, Skin lesion and Nuclei segmentation. The authors believe that the ability of USegTransformer-P and USegTransformer-S to perform segmentation with high precision could remarkably benefit medical practitioners and radiologists around the world.

Artificial Intelligence in the Diagnosis of Hepatocellular Carcinoma: A Systematic Review

Hepatocellular carcinoma ranks fifth amongst the most common malignancies and is the third most common cause of cancer-related death globally. Artificial Intelligence is a rapidly growing field of interest. Following the PRISMA reporting guidelines, we conducted a systematic review to retrieve articles reporting the application of AI in HCC detection and characterization. A total of 27 articles were included and analyzed with our composite score for the evaluation of the quality of the publications. The contingency table reported a statistically significant constant improvement over the years of the total quality score (p = 0.004). Different AI methods have been adopted in the included articles correlated with 19 articles studying CT (41.30%), 20 studying US (43.47%), and 7 studying MRI (15.21%). No article has discussed the use of artificial intelligence in PET and X-ray technology. Our systematic approach has shown that previous works in HCC detection and characterization have assessed the comparability of conventional interpretation with machine learning using US, CT, and MRI. The distribution of the imaging techniques in our analysis reflects the usefulness and evolution of medical imaging for the diagnosis of HCC. Moreover, our results highlight an imminent need for data sharing in collaborative data repositories to minimize unnecessary repetition and wastage of resources.

The feasibility to use artificial intelligence to aid detecting focal liver lesions in real-time ultrasound: a preliminary study based on videos

Despite the wide availability of ultrasound machines for hepatocellular carcinoma surveillance, an inadequate number of expert radiologists performing ultrasounds in remote areas remains a primary barrier for surveillance. We demonstrated feasibility of artificial intelligence (AI) to aid in the detection of focal liver lesions (FLLs) during ultrasound. An AI system for FLL detection in ultrasound videos was developed. Data in this study were prospectively collected at a university hospital. We applied a two-step training strategy for developing the AI system by using a large collection of ultrasound snapshot images and frames from full-length ultrasound videos. Detection performance of the AI system was evaluated and then compared to detection performance by 25 physicians including 16 non-radiologist physicians and 9 radiologists. Our dataset contained 446 videos (273 videos with 387 FLLs and 173 videos without FLLs) from 334 patients. The videos yielded 172,035 frames with FLLs and 1,427,595 frames without FLLs for training on the AI system. The AI system achieved an overall detection rate of 89.8% (95%CI: 84.5-95.0) which was significantly higher than that achieved by non-radiologist physicians (29.1%, 95%CI: 21.2-37.0, p < 0.001) and radiologists (70.9%, 95%CI: 63.0-78.8, p < 0.001). Median false positive detection rate by the AI system was 0.7% (IQR: 1.3%). AI system operation speed reached 30-34 frames per second, showing real-time feasibility. A further study to demonstrate whether the AI system can assist operators during ultrasound examinations is warranted.

Non-invasive multi-channel deep learning convolutional neural networks for localization and classification of common hepatic lesions

Purpose

Machine learning techniques, especially convolutional neural networks (CNN), have revolutionized the spectrum of computer vision tasks with a primary focus on supervised and labelled image datasets. We aimed to assess a novel method to segment the liver from the abdomen computed tomography (CT) image using the CNN network, and to train a unique method to locate and classify liver lesion pre-histological findings using multi-channel deep learning CNN (MDL-CNN).

Material and methods

The post-contrast CT images of the liver with a resolution of 0.625 mm were chosen for the study. In a random method, 50 examples of each hepatocellular carcinomas, metastases tumours, haemangiomas, hepatic cysts were chosen and evaluated.

Results

The dice score quantitatively analyses the similarity of segmentation results with the training dataset. In the first CNN model for segmenting the liver, the dice score was 96.18%. The MDL-CNN model yielded 98.78% accuracy in classification, and the dice score for locating liver lesions was 95.70%. Additionally, the performance of this model was compared to various other existing models.

Conclusions

According to our study, the machine learning approach can be successfully implemented to segment the liver and classify lesions, which will help radiologists impart better diagnosis.

Classification of focal liver lesions in CT images using convolutional neural networks with lesion information augmented patches and synthetic data augmentation

PURPOSE

We propose a deep learning method that classifies focal liver lesions (FLLs) into cysts, hemangiomas, and metastases from portal phase abdominal CT images. We propose a synthetic data augmentation process to alleviate the class imbalance and the Lesion INformation Augmented (LINA) patch to improve the learning efficiency.

METHODS

A dataset of 502 portal phase CT scans of 1,290 FLLs was used. First, to alleviate the class imbalance and to diversify the training data patterns, we suggest synthetic training data augmentation using DCGAN-based lesion mask synthesis and pix2pix-based mask-to-image translation. Second, to improve the learning efficiency of convolutional neural networks (CNNs) for the small lesions, we propose a novel type of input patch termed the LINA patch to emphasize the lesion texture information while also maintaining the lesion boundary information in the patches. Third, we construct a multi-scale CNN through a model ensemble of ResNet-18 CNNs trained on LINA patches of various mini-patch sizes.

RESULTS

The experiments demonstrate that (a) synthetic data augmentation method shows characteristics different but complementary to those in conventional real data augmentation in augmenting data distributions, (b) the proposed LINA patches improve classification performance compared to those by existing types of CNN input patches due to the enhanced texture and boundary information in the small lesions, and (c) through an ensemble of LINA patch-trained CNNs with different mini-patch sizes, the multi-scale CNN further improves overall classification performance. As a result, the proposed method achieved an accuracy of 87.30%, showing improvements of 10.81%p and 15.0%p compared to the conventional image patch-trained CNN and texture feature-trained SVM, respectively.

CONCLUSIONS

The proposed synthetic data augmentation method shows promising results in improving the data diversity and class imbalance, and the proposed LINA patches enhance the learning efficiency compared to the existing input image patches.

Discovering Digital Tumor Signatures-Using Latent Code Representations to Manipulate and Classify Liver Lesions

Simple Summary

We use a generative deep learning paradigm for the identification of digital signatures in radiological imaging data. The model is trained on a small inhouse data set and evaluated on publicly available data. Apart from using the learned signatures for the characterization of lesions, in analogy to radiomics features, we also demonstrate that by manipulating them we can create realistic synthetic CT image patches. This generation of synthetic data can be carried out at user-defined spatial locations. Moreover, the discrimination of liver lesions from normal liver tissue can be achieved with high accuracy, sensitivity, and specificity.

Abstract

Modern generative deep learning (DL) architectures allow for unsupervised learning of latent representations that can be exploited in several downstream tasks. Within the field of oncological medical imaging, we term these latent representations “digital tumor signatures” and hypothesize that they can be used, in analogy to radiomics features, to differentiate between lesions and normal liver tissue. Moreover, we conjecture that they can be used for the generation of synthetic data, specifically for the artificial insertion and removal of liver tumor lesions at user-defined spatial locations in CT images. Our approach utilizes an implicit autoencoder, an unsupervised model architecture that combines an autoencoder and two generative adversarial network (GAN)-like components. The model was trained on liver patches from 25 or 57 inhouse abdominal CT scans, depending on the experiment, demonstrating that only minimal data is required for synthetic image generation. The model was evaluated on a publicly available data set of 131 scans. We show that a PCA embedding of the latent representation captures the structure of the data, providing the foundation for the targeted insertion and removal of tumor lesions. To assess the quality of the synthetic images, we conducted two experiments with five radiologists. For experiment 1, only one rater and the ensemble-rater were marginally above the chance level in distinguishing real from synthetic data. For the second experiment, no rater was above the chance level. To illustrate that the “digital signatures” can also be used to differentiate lesion from normal tissue, we employed several machine learning methods. The best performing method, a LinearSVM, obtained 95% (97%) accuracy, 94% (95%) sensitivity, and 97% (99%) specificity, depending on if all data or only normal appearing patches were used for training of the implicit autoencoder. Overall, we demonstrate that the proposed unsupervised learning paradigm can be utilized for the removal and insertion of liver lesions at user defined spatial locations and that the digital signatures can be used to discriminate between lesions and normal liver tissue in abdominal CT scans.

Radiomics analysis of contrast-enhanced CT for classification of hepatic focal lesions in colorectal cancer patients: its limitations compared to radiologists

OBJECTIVE

To evaluate diagnostic performance of a radiomics model for classifying hepatic cyst, hemangioma, and metastasis in patients with colorectal cancer (CRC) from portal-phase abdominopelvic CT images.

METHODS

This retrospective study included 502 CRC patients who underwent contrast-enhanced CT and contrast-enhanced liver MRI between January 2005 and December 2010. Portal-phase CT images of training (n = 386) and validation (n = 116) cohorts were used to develop a radiomics model for differentiating three classes of liver lesions. Among multiple handcrafted features, the feature selection was performed using ReliefF method, and random forest classifiers were used to train the selected features. Diagnostic performance of the developed model was compared with that of four radiologists. A subgroup analysis was conducted based on lesion size.

RESULTS

The radiomics model demonstrated significantly lower overall and hemangioma- and metastasis-specific polytomous discrimination index (PDI) (overall, 0.8037; hemangioma-specific, 0.6653; metastasis-specific, 0.8027) than the radiologists (overall, 0.9622-0.9680; hemangioma-specific, 0.9452-0.9630; metastasis-specific, 0.9511-0.9869). For subgroup analysis, the PDI of the radiomics model was different according to the lesion size (< 10 mm, 0.6486; ≥ 10 mm, 0.8264) while that of the radiologists was relatively maintained. For classifying metastasis from benign lesions, the radiomics model showed excellent diagnostic performance, with an accuracy of 84.36% and an AUC of 0.9426.

CONCLUSION

Albeit inferior to the radiologists, the radiomics model achieved substantial diagnostic performance when differentiating hepatic lesions from portal-phase CT images of CRC patients. This model was limited particularly to classifying hemangiomas and subcentimeter lesions.

KEY POINTS

• Albeit inferior to the radiologists, the radiomics model could differentiate cyst, hemangioma, and metastasis with substantial diagnostic performance using portal-phase CT images of colorectal cancer patients. • The radiomics model demonstrated limitations especially in classifying hemangiomas and subcentimeter liver lesions.

Computer-aided diagnosis of liver lesions using CT images: A systematic review

BACKGROUND

Medical image processing has a strong footprint in radio diagnosis for the detection of diseases from the images. Several computer-aided systems were researched in the recent past to assist the radiologist in diagnosing liver diseases and reducing the interpretation time. The aim of this paper is to provide an overview of the state-of-the-art techniques in computer-assisted diagnosis systems to predict benign and malignant lesions using computed tomography images.

METHODS

The research articles published between 1998 and 2020 obtained from various standard databases were considered for preparing the review. The research papers include both conventional as well as deep learning-based systems for liver lesion diagnosis. The paper initially discusses the various hepatic lesions that are identifiable on computed tomography images, then the computer-aided diagnosis systems and their workflow. The conventional and deep learning-based systems are presented in stages wherein the various methods used for preprocessing, liver and lesion segmentation, radiological feature extraction and classification are discussed.

CONCLUSION

The review suggests the scope for future, work as efficient and effective segmentation methods that work well with diverse images have not been developed. Furthermore, unsupervised and semi-supervised deep learning models were not investigated for liver disease diagnosis in the reviewed papers. Other areas to be explored include image fusion and inclusion of essential clinical features along with the radiological features for better classification accuracy.

Optimization of FireNet for Liver Lesion Classification

In recent years, deep learning techniques, and in particular convolutional neural networks (CNNs) methods have demonstrated a superior performance in image classification and visual object recognition. In this work, we propose a classification of four types of liver lesions, namely, hepatocellular carcinoma, metastases, hemangiomas, and healthy tissues using convolutional neural networks with a succinct model called FireNet. We improved speed for quick classification and decreased the model size and the number of parameters by using fire modules from SqueezeNet. We have used bypass connection by adding it around Fire modules for learning a residual function between input and output, and to solve the vanishing gradient problem. We have proposed a new Particle Swarm Optimization (NPSO) to optimize the network parameters in order to further boost the performance of the proposed FireNet. The experimental results show that the parameters of FireNet are 9.5 times smaller than GoogLeNet, 51.6 times smaller than AlexNet, and 75.8 smaller than ResNet. The size of FireNet is reduced 16.6 times smaller than GoogLeNet, 75 times smaller than AlexNet and 76.6 times smaller than ResNet. The final accuracy of our proposed FireNet model was 89.2%.

Multiphase convolutional dense network for the classification of focal liver lesions on dynamic contrast-enhanced computed tomography

BACKGROUND

The accurate classification of focal liver lesions (FLLs) is essential to properly guide treatment options and predict prognosis. Dynamic contrast-enhanced computed tomography (DCE-CT) is still the cornerstone in the exact classification of FLLs due to its noninvasive nature, high scanning speed, and high-density resolution. Since their recent development, convolutional neural network-based deep learning techniques has been recognized to have high potential for image recognition tasks.

AIM

To develop and evaluate an automated multiphase convolutional dense network (MP-CDN) to classify FLLs on multiphase CT.

METHODS

A total of 517 FLLs scanned on a 320-detector CT scanner using a four-phase DCE-CT imaging protocol (including precontrast phase, arterial phase, portal venous phase, and delayed phase) from 2012 to 2017 were retrospectively enrolled. FLLs were classified into four categories: Category A, hepatocellular carcinoma (HCC); category B, liver metastases; category C, benign non-inflammatory FLLs including hemangiomas, focal nodular hyperplasias and adenomas; and category D, hepatic abscesses. Each category was split into a training set and test set in an approximate 8:2 ratio. An MP-CDN classifier with a sequential input of the four-phase CT images was developed to automatically classify FLLs. The classification performance of the model was evaluated on the test set; the accuracy and specificity were calculated from the confusion matrix, and the area under the receiver operating characteristic curve (AUC) was calculated from the SoftMax probability outputted from the last layer of the MP-CDN.

RESULTS

A total of 410 FLLs were used for training and 107 FLLs were used for testing. The mean classification accuracy of the test set was 81.3% (87/107). The accuracy/specificity of distinguishing each category from the others were 0.916/0.964, 0.925/0.905, 0.860/0.918, and 0.925/0.963 for HCC, metastases, benign non-inflammatory FLLs, and abscesses on the test set, respectively. The AUC (95% confidence interval) for differentiating each category from the others was 0.92 (0.837-0.992), 0.99 (0.967-1.00), 0.88 (0.795-0.955) and 0.96 (0.914-0.996) for HCC, metastases, benign non-inflammatory FLLs, and abscesses on the test set, respectively.

CONCLUSION

MP-CDN accurately classified FLLs detected on four-phase CT as HCC, metastases, benign non-inflammatory FLLs and hepatic abscesses and may assist radiologists in identifying the different types of FLLs.

Automated Detection and Segmentation of Early Gastric Cancer from Endoscopic Images Using Mask R-CNN

Gastrointestinal endoscopy is widely conducted for the early detection of gastric cancer. However, it is often difficult to detect early gastric cancer lesions and accurately evaluate the invasive regions. Our study aimed to develop a detection and segmentation method for early gastric cancer regions from gastrointestinal endoscopic images. In this method, we first collected 1208 healthy and 533 cancer images. The gastric cancer region was detected and segmented from endoscopic images using Mask R-CNN, an instance segmentation method. An endoscopic image was provided to the Mask R-CNN, and a bounding box and a label image of the gastric cancer region were obtained. As a performance evaluation via five-fold cross-validation, sensitivity and false positives (FPs) per image were 96.0% and 0.10 FP/image, respectively. In the evaluation of segmentation of the gastric cancer region, the average Dice index was 71%. These results indicate that our proposed scheme may be useful for the detection of gastric cancer and evaluation of the invasive region in gastrointestinal endoscopy.

Optimizing deep belief network parameters using grasshopper algorithm for liver disease classification

Image processing plays a vital role in many areas such as healthcare, military, scientific and business due to its wide variety of advantages and applications. Detection of computed tomography (CT) liver disease is one of the difficult tasks in the medical field. Hand crafted features and classifications are the two types of methods used in the previous approaches, to classify liver disease. But these classification results are not optimal. In this article, we propose a novel method utilizing deep belief network (DBN) with grasshopper optimization algorithm (GOA) for liver disease classification. Initially, the image quality is enhanced by preprocessing techniques and then features like texture, color and shape are extracted. The extracted features are reduced by utilizing the dimensionality reduction method like principal component analysis (PCA). Here, the DBN parameters are optimized using GOA for recognizing liver disease. The experiments are performed on the real time and open source CT image datasets which embraces normal, cyst, hepatoma, and cavernous hemangiomas, fatty liver, metastasis, cirrhosis, and tumor samples. The proposed method yields 98% accuracy, 95.82% sensitivity, 97.52% specificity, 98.53% precision, and 96.8% F‐1 score in simulation process when compared with other existing techniques.

A Stacked Fully Convolutional Networks with Feature Alignment Framework for Multi-Label Land-cover Segmentation

Applying deep-learning methods, especially fully convolutional networks (FCNs), has become a popular option for land-cover classification or segmentation in remote sensing. Compared with traditional solutions, these approaches have shown promising generalization capabilities and precision levels in various datasets of different scales, resolutions, and imaging conditions. To achieve superior performance, a lot of research has focused on constructing more complex or deeper networks. However, using an ensemble of different fully convolutional models to achieve better generalization and to prevent overfitting has long been ignored. In this research, we design four stacked fully convolutional networks (SFCNs), and a feature alignment framework for multi-label land-cover segmentation. The proposed feature alignment framework introduces an alignment loss of features extracted from basic models to balance their similarity and variety. Experiments on a very high resolution(VHR) image dataset with six categories of land-covers indicates that the proposed SFCNs can gain better performance when compared to existing deep learning methods. In the 2nd variant of SFCN, the optimal feature alignment gains increments of 4.2% (0.772 vs. 0.741), 6.8% (0.629 vs. 0.589), and 5.5% (0.727 vs. 0.689) for its f1-score, jaccard index, and kappa coefficient, respectively.

Chronic Liver Disease Classification Using Hybrid Whale Optimization with Simulated Annealing and Ensemble Classifier

Chronic liver disease (CLD), which indicates the inflammatory condition of the liver, leads to cirrhosis or even partial or total liver dysfunction when left untreated. A non-invasive approach for evaluating CLD with computed tomography (CT) images is proposed using an ensemble of classifiers. To accurately classify CLD, the hybrid whale optimization algorithm with simulated annealing (WOA-SA) is used in selecting an optimal set of features. The proposed method employs seven sets of features with a total of 73–3D (three-dimensional) texture features. A hybrid ensemble classifier with support vector machine (SVM), k—Nearest Neighbor (k-NN), and random forest (RF) classifiers are used to classify liver diseases. Experimental analysis is performed on clinical CT images datasets, which include normal liver, fatty liver, metastasis, cirrhosis, and cancerous samples. The optimal features selected using the WOA-SA improve the accuracy of CLD classification for the five classes of diseases mentioned above. The accuracy of the liver classification using ensemble classifier yields approximately 98% with a 95% confidence interval (CI) of (0.7789, 1.0000) and an error rate of 1.9%. The performance of the proposed method is compared with two existing algorithms and the sensitivity and specificity yield an overall average of 96% and 93%, with 95% confidence interval of (0.7513, 1.0000) and (0.7126, 1.0000), respectively. Classification of CLD based on ensemble classifier illustrates the effectiveness of the proposed method and the comparison analysis demonstrates the superiority of the methodology.

Combined Volumetric and Density Analyses of Contrast-Enhanced CT Imaging to Assess Drug Therapy Response in Gastroenteropancreatic Neuroendocrine Diffuse Liver Metastasis

OBJECTIVE

We propose a computer-aided method to assess response to drug treatment, using CT imaging-based volumetric and density measures in patients with gastroenteropancreatic neuroendocrine tumors (GEP-NETs) and diffuse liver metastases.

METHODS

Twenty-five patients with GEP-NETs with diffuse liver metastases were enrolled. Pre- and posttreatment CT examinations were retrospectively analyzed. Total tumor volume (volume) and mean volumetric tumor density (density) were calculated based on tumor segmentation on CT images. The maximum axial diameter (tumor size) for each target tumor was measured on pre- and posttreatment CT images according to Response Evaluation Criteria In Solid Tumors (RECIST). Progression-free survival (PFS) for each patient was measured and recorded.

RESULTS

Correlation analysis showed inverse correlation between change of volume and density (Δ(V + D)), change of volume (ΔV), and change of tumor size (ΔS) with PFS (r = -0.653, P=0.001; r = -0.617, P=0.003; r = -0.548, P=0.01, respectively). There was no linear correlation between ΔD and PFS (r = -0.226, P=0.325).

CONCLUSION

The changes of volume and density derived from CT images of all lesions showed a good correlation with PFS and may help assess treatment response.

Virtual monoenergetic imaging in rapid kVp-switching dual-energy CT (DECT) of the abdomen: impact on CT texture analysis

PURPOSE

To study the impact of keV levels of virtual monoenergetic images generated from rapid kVp-switching dual-energy CT (rsDECT) on CT texture analysis (CTTA).

METHODS

This study included 30 consecutive patients (59.3 ± 12 years; range 34-77 years; 17M:13F) who underwent portal venous phase abdominal CT on a rsDECT scanner. Axial 5-mm monoenergetic images at 5 energy levels (40/50/60/70/80 keV) were created and CTTA of liver was performed. CTTA comprised a filtration-histogram technique with different spatial scale filter (SSF) values (0-6). CTTA quantification at each SSF value included histogram-based statistical parameters such as mean intensity, standard deviation (SD), entropy, mean of positive pixels (MPP), skewness, and kurtosis. The values were compared using repeated measures ANOVA.

RESULTS

Among the different CTTA metrics, mean intensity (at SSF > 0), skewness, and kurtosis did not show variability whereas entropy, MPP, and SD varied with different keV levels. There was no change in skewness and kurtosis values for all 6 filters (p > 0.05). Mean intensity showed no change for filters 2-6 (p > 0.05). Mean intensity at SSF = 0 i.e., mean attenuations were 91.2 ± 2.9, 108.7 ± 3.6, 136.1 ± 4.7, 179.8 ± 6.9, and 250.5 ± 10.1 HU for 80, 70, 60, 50, and 40 keV images, respectively demonstrating significant variability (decrease) with increasing keV levels (p < 0.001). Entropy, MPP, and SD values showed a statistically significant decrease with increasing keV of monoenergetic images on all 6 filters (p < 0.001).

CONCLUSION

The energy levels of monoenergetic images have variable impact on the different CTTA parameters, with no significant change in skewness, kurtosis, and filtered mean intensity whereas significant decrease in mean attenuation, entropy, MPP, and SD values with increasing energy levels.

Spatial Bayesian modeling of GLCM with application to malignant lesion characterization

The emerging field of cancer radiomics endeavors to characterize intrinsic patterns of tumor phenotypes and surrogate markers of response by transforming medical images into objects that yield quantifiable summary statistics to which regression and machine learning algorithms may be applied for statistical interrogation. Recent literature has identified clinicopathological association based on textural features deriving from gray-level co-occurrence matrices (GLCM) which facilitate evaluations of gray-level spatial dependence within a delineated region of interest. GLCM-derived features, however, tend to contribute highly redundant information. Moreover, when reporting selected feature sets, investigators often fail to adjust for multiplicities and commonly fail to convey the predictive power of their findings. This article presents a Bayesian probabilistic modeling framework for the GLCM as a multivariate object as well as describes its application within a cancer detection context based on computed tomography. The methodology, which circumvents processing steps and avoids evaluations of reductive and highly correlated feature sets, uses latent Gaussian Markov random field structure to characterize spatial dependencies among GLCM cells and facilitates classification via predictive probability. Correctly predicting the underlying pathology of 81% of the adrenal lesions in our case study, the proposed method outperformed current practices which achieved a maximum accuracy of only 59%. Simulations and theory are presented to further elucidate this comparison as well as ascertain the utility of applying multivariate Gaussian spatial processes to GLCM objects.

Deep Learning with Convolutional Neural Network for Differentiation of Liver Masses at Dynamic Contrast-enhanced CT: A Preliminary Study

Purpose To investigate diagnostic performance by using a deep learning method with a convolutional neural network (CNN) for the differentiation of liver masses at dynamic contrast agent-enhanced computed tomography (CT). Materials and Methods This clinical retrospective study used CT image sets of liver masses over three phases (noncontrast-agent enhanced, arterial, and delayed). Masses were diagnosed according to five categories (category A, classic hepatocellular carcinomas [HCCs]; category B, malignant liver tumors other than classic and early HCCs; category C, indeterminate masses or mass-like lesions [including early HCCs and dysplastic nodules] and rare benign liver masses other than hemangiomas and cysts; category D, hemangiomas; and category E, cysts). Supervised training was performed by using 55 536 image sets obtained in 2013 (from 460 patients, 1068 sets were obtained and they were augmented by a factor of 52 [rotated, parallel-shifted, strongly enlarged, and noise-added images were generated from the original images]). The CNN was composed of six convolutional, three maximum pooling, and three fully connected layers. The CNN was tested with 100 liver mass image sets obtained in 2016 (74 men and 26 women; mean age, 66.4 years ± 10.6 [standard deviation]; mean mass size, 26.9 mm ± 25.9; 21, nine, 35, 20, and 15 liver masses for categories A, B, C, D, and E, respectively). Training and testing were performed five times. Accuracy for categorizing liver masses with CNN model and the area under receiver operating characteristic curve for differentiating categories A-B versus categories C-E were calculated. Results Median accuracy of differential diagnosis of liver masses for test data were 0.84. Median area under the receiver operating characteristic curve for differentiating categories A-B from C-E was 0.92. Conclusion Deep learning with CNN showed high diagnostic performance in differentiation of liver masses at dynamic CT. ^© RSNA, 2017 Online supplemental material is available for this article.

Novel Mahalanobis-based feature selection improves one-class classification of early hepatocellular carcinoma

Detection of early hepatocellular carcinoma (HCC) is responsible for increasing survival rates in up to 40%. One-class classifiers can be used for modeling early HCC in multidetector computed tomography (MDCT), but demand the specific knowledge pertaining to the set of features that best describes the target class. Although the literature outlines several features for characterizing liver lesions, it is unclear which is most relevant for describing early HCC. In this paper, we introduce an unconstrained GA feature selection algorithm based on a multi-objective Mahalanobis fitness function to improve the classification performance for early HCC. We compared our approach to a constrained Mahalanobis function and two other unconstrained functions using Welch's t-test and Gaussian Data Descriptors. The performance of each fitness function was evaluated by cross-validating a one-class SVM. The results show that the proposed multi-objective Mahalanobis fitness function is capable of significantly reducing data dimensionality (96.4%) and improving one-class classification of early HCC (0.84 AUC). Furthermore, the results provide strong evidence that intensity features extracted at the arterial to portal and arterial to equilibrium phases are important for classifying early HCC.

Large-scale retrieval for medical image analytics: A comprehensive review

Over the past decades, medical image analytics was greatly facilitated by the explosion of digital imaging techniques, where huge amounts of medical images were produced with ever-increasing quality and diversity. However, conventional methods for analyzing medical images have achieved limited success, as they are not capable to tackle the huge amount of image data. In this paper, we review state-of-the-art approaches for large-scale medical image analysis, which are mainly based on recent advances in computer vision, machine learning and information retrieval. Specifically, we first present the general pipeline of large-scale retrieval, summarize the challenges/opportunities of medical image analytics on a large-scale. Then, we provide a comprehensive review of algorithms and techniques relevant to major processes in the pipeline, including feature representation, feature indexing, searching, etc. On the basis of existing work, we introduce the evaluation protocols and multiple applications of large-scale medical image retrieval, with a variety of exploratory and diagnostic scenarios. Finally, we discuss future directions of large-scale retrieval, which can further improve the performance of medical image analysis.

Medical Image Retrieval Using Multi-Texton Assignment

In this paper, we present a multi-texton representation method for medical image retrieval, which utilizes the locality constraint to encode each filter bank response within its local-coordinate system consisting of the k nearest neighbors in texton dictionary and subsequently employs spatial pyramid matching technique to implement feature vector representation. Comparison with the traditional nearest neighbor assignment followed by texton histogram statistics method, our strategies reduce the quantization errors in mapping process and add information about the spatial layout of texton distributions and, thus, increase the descriptive power of the image representation. We investigate the effects of different parameters on system performance in order to choose the appropriate ones for our datasets and carry out experiments on the IRMA-2009 medical collection and the mammographic patch dataset. The extensive experimental results demonstrate that the proposed method has superior performance.

Computer-aided diagnosis of liver tumors on computed tomography images

BACKGROUND AND OBJECTIVE

Liver cancer is the tenth most common cancer in the USA, and its incidence has been increasing for several decades. Early detection, diagnosis, and treatment of the disease are very important. Computed tomography (CT) is one of the most common and robust imaging techniques for the detection of liver cancer. CT scanners can provide multiple-phase sequential scans of the whole liver. In this study, we proposed a computer-aided diagnosis (CAD) system to diagnose liver cancer using the features of tumors obtained from multiphase CT images.

METHODS

A total of 71 histologically-proven liver tumors including 49 benign and 22 malignant lesions were evaluated with the proposed CAD system to evaluate its performance. Tumors were identified by the user and then segmented using a region growing algorithm. After tumor segmentation, three kinds of features were obtained for each tumor, including texture, shape, and kinetic curve. The texture was quantified using 3 dimensional (3-D) texture data of the tumor based on the grey level co-occurrence matrix (GLCM). Compactness, margin, and an elliptic model were used to describe the 3-D shape of the tumor. The kinetic curve was established from each phase of tumor and represented as variations in density between each phase. Backward elimination was used to select the best combination of features, and binary logistic regression analysis was used to classify the tumors with leave-one-out cross validation.

RESULTS

The accuracy and sensitivity for the texture were 71.82% and 68.18%, respectively, which were better than for the shape and kinetic curve under closed specificity. Combining all of the features achieved the highest accuracy (58/71, 81.69%), sensitivity (18/22, 81.82%), and specificity (40/49, 81.63%). The Az value of combining all features was 0.8713.

CONCLUSIONS

Combining texture, shape, and kinetic curve features may be able to differentiate benign from malignant tumors in the liver using our proposed CAD system.

Detection of prostate cancer in multiparametric MRI using random forest with instance weighting

A prostate computer-aided diagnosis (CAD) based on random forest to detect prostate cancer using a combination of spatial, intensity, and texture features extracted from three sequences, T2W, ADC, and B2000 images, is proposed. The random forest training considers instance-level weighting for equal treatment of small and large cancerous lesions as well as small and large prostate backgrounds. Two other approaches, based on an AutoContext pipeline intended to make better use of sequence-specific patterns, were considered. One pipeline uses random forest on individual sequences while the other uses an image filter described to produce probability map-like images. These were compared to a previously published CAD approach based on support vector machine (SVM) evaluated on the same data. The random forest, features, sampling strategy, and instance-level weighting improve prostate cancer detection performance [area under the curve (AUC) 0.93] in comparison to SVM (AUC 0.86) on the same test data. Using a simple image filtering technique as a first-stage detector to highlight likely regions of prostate cancer helps with learning stability over using a learning-based approach owing to visibility and ambiguity of annotations in each sequence.

Weighted locality-constrained linear coding for lesion classification in CT images

Computed tomography is a popular imaging modality for detecting abnormalities associated with abdominal organs such as the liver, kidney and uterus. In this paper, we propose a novel weighted locality-constrained linear coding (LLC) method followed by a weighted max-pooling method to classify liver lesions into three classes: cysts, metastases, hemangiomas. We first divide the lesions into same-size patches. Then, we extract the raw features in all patches followed by Principal Components Analysis (PCA) and apply K means to obtain a single LLC dictionary. Since the interior lesion patches and the boundary patches contribute different information in the image, we assign different weights on these two types of patches to obtain the LLC codes. Moreover, a weighted max pooling approach is also proposed to further evaluate the importance of these two types of patches in feature pooling. Experiments on 109 images of liver lesions were carried out to validate the proposed method. The proposed method achieves a best lesion classification accuracy of 96.33%, which appears to be superior compared with traditional image coding methods: LLC method and Bag-of-words method (BoW) and traditional features: Local Binary Pattern (LBP) features, uniform LBP and complete LBP, demonstrating that the proposed method provides better classification.

Radiomics: a new application from established techniques

The increasing use of biomarkers in cancer have led to the concept of personalized medicine for patients. Personalized medicine provides better diagnosis and treatment options available to clinicians. Radiological imaging techniques provide an opportunity to deliver unique data on different types of tissue. However, obtaining useful information from all radiological data is challenging in the era of "big data". Recent advances in computational power and the use of genomics have generated a new area of research termed Radiomics. Radiomics is defined as the high throughput extraction of quantitative imaging features or texture (radiomics) from imaging to decode tissue pathology and creating a high dimensional data set for feature extraction. Radiomic features provide information about the gray-scale patterns, inter-pixel relationships. In addition, shape and spectral properties can be extracted within the same regions of interest on radiological images. Moreover, these features can be further used to develop computational models using advanced machine learning algorithms that may serve as a tool for personalized diagnosis and treatment guidance.

Improved Patch-Based Automated Liver Lesion Classification by Separate Analysis of the Interior and Boundary Regions

The bag-of-visual-words (BoVW) method with construction of a single dictionary of visual words has been used previously for a variety of classification tasks in medical imaging, including the diagnosis of liver lesions. In this paper, we describe a novel method for automated diagnosis of liver lesions in portal-phase computed tomography (CT) images that improves over single-dictionary BoVW methods by using an image patch representation of the interior and boundary regions of the lesions. Our approach captures characteristics of the lesion margin and of the lesion interior by creating two separate dictionaries for the margin and the interior regions of lesions ("dual dictionaries" of visual words). Based on these dictionaries, visual word histograms are generated for each region of interest within the lesion and its margin. For validation of our approach, we used two datasets from two different institutions, containing CT images of 194 liver lesions (61 cysts, 80 metastasis, and 53 hemangiomas). The final diagnosis of each lesion was established by radiologists. The classification accuracy for the images from the two institutions was 99% and 88%, respectively, and 93% for a combined dataset. Our new BoVW approach that uses dual dictionaries shows promising results. We believe the benefits of our approach may generalize to other application domains within radiology.

Quantification of heterogeneity as a biomarker in tumor imaging: a systematic review

BACKGROUND

Many techniques are proposed for the quantification of tumor heterogeneity as an imaging biomarker for differentiation between tumor types, tumor grading, response monitoring and outcome prediction. However, in clinical practice these methods are barely used. This study evaluates the reported performance of the described methods and identifies barriers to their implementation in clinical practice.

METHODOLOGY

The Ovid, Embase, and Cochrane Central databases were searched up to 20 September 2013. Heterogeneity analysis methods were classified into four categories, i.e., non-spatial methods (NSM), spatial grey level methods (SGLM), fractal analysis (FA) methods, and filters and transforms (F&T). The performance of the different methods was compared.

PRINCIPAL FINDINGS

Of the 7351 potentially relevant publications, 209 were included. Of these studies, 58% reported the use of NSM, 49% SGLM, 10% FA, and 28% F&T. Differentiation between tumor types, tumor grading and/or outcome prediction was the goal in 87% of the studies. Overall, the reported area under the curve (AUC) ranged from 0.5 to 1 (median 0.87). No relation was found between the performance and the quantification methods used, or between the performance and the imaging modality. A negative correlation was found between the tumor-feature ratio and the AUC, which is presumably caused by overfitting in small datasets. Cross-validation was reported in 63% of the classification studies. Retrospective analyses were conducted in 57% of the studies without a clear description.

CONCLUSIONS

In a research setting, heterogeneity quantification methods can differentiate between tumor types, grade tumors, and predict outcome and monitor treatment effects. To translate these methods to clinical practice, more prospective studies are required that use external datasets for validation: these datasets should be made available to the community to facilitate the development of new and improved methods.

Artificial neural network application in the diagnosis of disease conditions with liver ultrasound images

The preliminary study presented within this paper shows a comparative study of various texture features extracted from liver ultrasonic images by employing Multilayer Perceptron (MLP), a type of artificial neural network, to study the presence of disease conditions. An ultrasound (US) image shows echo-texture patterns, which defines the organ characteristics. Ultrasound images of liver disease conditions such as "fatty liver," "cirrhosis," and "hepatomegaly" produce distinctive echo patterns. However, various ultrasound imaging artifacts and speckle noise make these echo-texture patterns difficult to identify and often hard to distinguish visually. Here, based on the extracted features from the ultrasonic images, we employed an artificial neural network for the diagnosis of disease conditions in liver and finding of the best classifier that distinguishes between abnormal and normal conditions of the liver. Comparison of the overall performance of all the feature classifiers concluded that "mixed feature set" is the best feature set. It showed an excellent rate of accuracy for the training data set. The gray level run length matrix (GLRLM) feature shows better results when the network was tested against unknown data.

A Hybrid Approach to Diagnosis of Hepatic Tumors in Computed Tomography Images

Liver cancer is one of the most popular cancer diseases and causes a large amount of death every year, can be reduced by early detection and diagnosis. Computer-aided liver analysis can help in the early detection and diagnosis of liver cancer. In this paper, enhancement and segmentation process is applied to increase the computation and focus on liver parenchyma. This parenchyma also segmented using Watershed and Region Growing algorithms to extract liver tumors. These tumors will be analyzed and characterized to distinguish between hemangioma (benign) and hepatocellular (malignant) tumors using Local Binary Pattern (LBP), Gray Level Co-occurrence matrix (GLCM), Fractal Dimension (FD) and feature fusion technique is applied to maximize and enhance the performance of the classifier rate. The authors review different methods for liver segmentation and abnormality classification. An attempt was made to combine the individual scores from different techniques in order to compensate their individual weaknesses and to preserve their strength. The authors present and exhaustively evaluate algorithms using computer vision techniques. The experimental results based on confusion matrix and kappa coefficient show that the higher accuracy is obtained of automatic agreement classification and suggest that the developed CAD system has great potential and promise in the automatic diagnosis of both benign and malignant tumors of liver.

Three-dimensional spatiotemporal features for fast content-based retrieval of focal liver lesions

Content-based image retrieval systems for 3-D medical datasets still largely rely on 2-D image-based features extracted from a few representative slices of the image stack. Most 2 -D features that are currently used in the literature not only model a 3-D tumor incompletely but are also highly expensive in terms of computation time, especially for high-resolution datasets. Radiologist-specified semantic labels are sometimes used along with image-based 2-D features to improve the retrieval performance. Since radiological labels show large interuser variability, are often unstructured, and require user interaction, their use as lesion characterizing features is highly subjective, tedious, and slow. In this paper, we propose a 3-D image-based spatiotemporal feature extraction framework for fast content-based retrieval of focal liver lesions. All the features are computer generated and are extracted from four-phase abdominal CT images. Retrieval performance and query processing times for the proposed framework is evaluated on a database of 44 hepatic lesions comprising of five pathological types. Bull's eye percentage score above 85% is achieved for three out of the five lesion pathologies and for 98% of query lesions, at least one same type of lesion is ranked among the top two retrieved results. Experiments show that the proposed system's query processing is more than 20 times faster than other already published systems that use 2-D features. With fast computation time and high retrieval accuracy, the proposed system has the potential to be used as an assistant to radiologists for routine hepatic tumor diagnosis.