Transfer learning with deep convolutional neural network for liver steatosis assessment in ultrasound images

Automatic diagnosis of abdominal pathologies in untrimmed ultrasound videos

PURPOSE

Despite major advances in Computer Assisted Diagnosis (CAD), the need for carefully labeled training data remains an important clinical translation barrier. This work aims to overcome this barrier for ultrasound video-based CAD, using video-level classification labels combined with a novel training strategy to improve the generalization performance of state-of-the-art (SOTA) video classifiers.

METHODS

SOTA video classifiers were trained and evaluated on a novel ultrasound video dataset of liver and kidney pathologies, and they all struggled to generalize, especially for kidney pathologies. A new training strategy is presented, wherein a frame relevance assessor is trained to score the video frames in a video by diagnostic relevance. This is used to automatically generate diagnostically-relevant video clips (DR-Clips), which guide a video classifier during training and inference.

RESULTS

Using DR-Clips with a Video Swin Transformer, we achieved a 0.92 ROC-AUC for kidney pathology detection in videos, compared to 0.72 ROC-AUC with a Swin Transformer and standard video clips. For liver steatosis detection, due to the diffuse nature of the pathology, the Video Swin Transformer, and other video classifiers, performed similarly well, generally exceeding a 0.92 ROC-AUC.

CONCLUSION

In theory, video classifiers, such as video transformers, should be able to solve ultrasound CAD tasks with video labels. However, in practice, video labels provide weaker supervision compared to image labels, resulting in worse generalization, as demonstrated. The additional frame guidance provided by DR-Clips enhances performance significantly. The results highlight current limits and opportunities to improve frame guidance.

Unsupervised Test-Time Adaptation for Hepatic Steatosis Grading Using Ultrasound B-Mode Images

Ultrasound (US) is considered a key modality for the clinical assessment of hepatic steatosis (i.e., fatty liver) due to its noninvasiveness and availability. Deep learning methods have attracted considerable interest in this field, as they are capable of learning patterns in a collection of images and achieve clinically comparable levels of accuracy in steatosis grading. However, variations in patient populations, acquisition protocols, equipment, and operator expertise across clinical sites can introduce domain shifts that reduce model performance when applied outside the original training setting. In response, unsupervised domain adaptation techniques are being investigated to address these shifts, allowing models to generalize more effectively across diverse clinical environments. In this work, we propose a test-time batch normalization (TTN) technique designed to handle domain shift, especially for changes in label distribution, by adapting selected features of batch normalization (BatchNorm) layers in a trained convolutional neural network model. This approach operates in an unsupervised manner, allowing robust adaptation to new distributions without access to label data. The method was evaluated on two abdominal US datasets collected at different institutions, assessing its capability in mitigating domain shift for hepatic steatosis classification. The proposed method reduced the mean absolute error in steatosis grading by 37% and improved the area under the receiver operating characteristic curves (AUC) for steatosis detection from 0.78 to 0.97, compared to nonadapted models. These findings demonstrate the potential of the proposed method to address domain shift in US-based hepatic steatosis diagnosis, minimizing risks associated with deploying trained models in various clinical settings.

Ultrasound detection of nonalcoholic steatohepatitis using convolutional neural networks with dual-branch global-local feature fusion architecture

Nonalcoholic steatohepatitis (NASH) is a contributing factor to liver cancer, with ultrasound B-mode imaging as the first-line diagnostic tool. This study applied deep learning to ultrasound B-scan images for NASH detection and introduced an ultrasound-specific data augmentation (USDA) technique with a dual-branch global-local feature fusion architecture (DG-LFFA) to improve model performance and adaptability across imaging conditions. A total of 137 participants were included. Ultrasound images underwent data augmentation (rotation and USDA) for training and testing convolutional neural networks-AlexNet, Inception V3, VGG16, VGG19, ResNet50, and DenseNet201. Gradient-weighted class activation mapping (Grad-CAM) analyzed model attention patterns, guiding the selection of the optimal backbone for DG-LFFA implementation. The models achieved testing accuracies of 0.81-0.83 with rotation-based data augmentation. Grad-CAM analysis showed that ResNet50 and DenseNet201 exhibited stronger liver attention. When USDA simulated datasets from different imaging conditions, DG-LFFA (based on ResNet50 and DenseNet201) improved accuracy (0.79 to 0.84 and 0.78 to 0.83), recall (0.72 to 0.81 and 0.70 to 0.78), and F1 score (0.80 to 0.84 for both models). In conclusion, deep architectures (ResNet50 and DenseNet201) enable focused analysis of liver regions for NASH detection. Under USDA-simulated imaging variations, the proposed DG-LFFA framework further improves diagnostic performance.

Automated ultrasonography of hepatocellular carcinoma using discrete wavelet transform based deep-learning neural network

This study introduces HCC-Net, a novel wavelet-based approach for the accurate diagnosis of hepatocellular carcinoma (HCC) from abdominal ultrasound (US) images using artificial neural networks. The HCC-Net integrates the discrete wavelet transform (DWT) to decompose US images into four sub-band images, a lesion detector for hierarchical lesion localization, and a pattern-augmented classifier for generating pattern-enhanced lesion images and subsequent classification. The lesion detection uses a hierarchical coarse-to-fine approach to minimize missed lesions. CoarseNet performs initial lesion localization, while FineNet identifies any lesions that were missed. In the classification phase, the wavelet components of detected lesions are synthesized to create pattern-augmented images that enhance feature distinction, resulting in highly accurate classifications. These augmented images are classified into 'Normal,' 'Benign,' or 'Malignant' categories according to their morphologic features on sonography. The experimental results demonstrate the significant effectiveness of the proposed coarse-to-fine detection framework and pattern-augmented classifier in lesion detection and classification. We achieved an accuracy of 96.2 %, a sensitivity of 97.6 %, and a specificity of 98.1 % on the Samsung Medical Center dataset, indicating HCC-Net's potential as a reliable tool for liver cancer screening.

The dietary patterns of water deer recently rediscovered in Northeast China exhibit remarkable similarities to those observed in other regions

Climate change, human activities, and habitat fragmentation-or even loss-pose ongoing threats to the survival of wildlife. Understanding the dietary habits of endangered species is a critical component of their conservation. In this study, we investigated the diet of water deer (Hydropotes inermis) by integrating traditional fecal microhistological analysis with a deep learning algorithm. Fecal samples were collected from water deer in northeastern China, with microscopic slides prepared for both the warm season (203 samples) and cold season (451 samples). A deep learning model was trained and tested using labeled images of 130 known plant species, achieving an accuracy of 99.83%. The results revealed that water deer consume 110 plant species from 86 genera and 40 families annually. The dietary patterns observed in this study align closely with those reported in other regions, reflecting species-specific foraging characteristics and further validating the reliability of deep learning algorithms in ecological research. Notably, significant seasonal variations were identified, highlighting the adaptability of water deer to changing environmental conditions. By examining the feeding ecology and seasonal dietary shifts of water deer in northeastern China, this study provides valuable insights for the development of targeted conservation strategies to support their populations in this region and beyond.

Mixture of Expert-Based SoftMax-Weighted Box Fusion for Robust Lesion Detection in Ultrasound Imaging

Background/Objectives: Ultrasound (US) imaging plays a crucial role in the early detection and treatment of hepatocellular carcinoma (HCC). However, challenges such as speckle noise, low contrast, and diverse lesion morphology hinder its diagnostic accuracy. Methods: To address these issues, we propose CSM-FusionNet, a novel framework that integrates clustering, SoftMax-weighted Box Fusion (SM-WBF), and padding. Using raw US images from a leading hospital, Samsung Medical Center (SMC), we applied intensity adjustment, adaptive histogram equalization, low-pass, and high-pass filters to reduce noise and enhance resolution. Data augmentation generated ten images per one raw US image, allowing the training of 10 YOLOv8 networks. The mAP@0.5 of each network was used as SoftMax-derived weights in SM-WBF. Threshold-lowered bounding boxes were clustered using Density-Based Spatial Clustering of Applications with Noise (DBSCAN), and outliers were managed within clusters. SM-WBF reduced redundant boxes, and padding enriched features, improving classification accuracy. Results: The accuracy improved from 82.48% to 97.58% with sensitivity reaching 100%. The framework increased lesion detection accuracy from 56.11% to 95.56% after clustering and SM-WBF. Conclusions: CSM-FusionNet demonstrates the potential to significantly improve diagnostic reliability in US-based lesion detection, aiding precise clinical decision-making.

Diagnostic of fatty liver using radiomics and deep learning models on non-contrast abdominal CT

PURPOSE

This study aims to explore the potential of non-contrast abdominal CT radiomics and deep learning models in accurately diagnosing fatty liver.

MATERIALS AND METHODS

The study retrospectively enrolled 840 individuals who underwent non-contrast abdominal CT and quantitative CT (QCT) examinations at the First Affiliated Hospital of Zhengzhou University from July 2022 to May 2023. Subsequently, these participants were divided into a training set (n = 539) and a testing set (n = 301) in a 9:5 ratio. The liver fat content measured by experienced radiologists using QCT technology served as the reference standard. The liver images from the non-contrast abdominal CT scans were then segmented as regions of interest (ROI) from which radiomics features were extracted. Two-dimensional (2D) and three-dimensional (3D) radiomics models, as well as 2D and 3D deep learning models, were developed, and machine learning models based on clinical data were constructed for the four-category diagnosis of fatty liver. The characteristic curves for each model were plotted, and area under the receiver operating characteristic curve (AUC) were calculated to assess their efficacy in the classification and diagnosis of fatty liver.

RESULTS

A total of 840 participants were included (mean age 49.1 years ± 11.5 years [SD]; 581 males), of whom 610 (73%) had fatty liver. Among the patients with fatty liver, there were 302 with mild fatty liver (CT fat fraction of 5%-14%), 155 with moderate fatty liver (CT fat fraction of 14%-28%), and 153 with severe fatty liver (CT fat fraction >28%). Among all models used for diagnosing fatty liver, the 2D radiomics model based on the random forest algorithm achieved the highest AUC (0.973), while the 2D radiomics model based on the Bagging decision tree algorithm showed the highest sensitivity (0.873), specificity (0.939), accuracy (0.864), precision (0.880), and F1 score (0.876).

CONCLUSION

A systematic comparison was conducted on the performance of 2D and 3D radiomics models, as well as deep learning models, in the diagnosis of four-category fatty liver. This comprehensive model comparison provides a broader perspective for determining the optimal model for liver fat diagnosis. It was found that the 2D radiomics models based on the random forest and Bagging decision tree algorithms show high consistency with the QCT-based classification diagnosis of fatty liver used by experienced radiologists.

Development of a Deep Learning Model for Classification of Hepatic Steatosis from Clinical Standard Ultrasound

OBJECTIVE

Early detection and monitoring of hepatic steatosis can help establish appropriate preventative measures against progression to more advanced disease. We aimed to develop a deep learning (DL) program for classification of hepatic steatosis from standard-of-care grayscale ultrasound (US) images.

METHODS

In this single-center retrospective study, we utilized grayscale US images from January 1, 2010, to October 23, 2022, labeled with magnetic resonance imaging (MRI) proton density fat fraction (MRI-PDFF) to develop a DL multi-instance program for differentiating normal (S0) from steatotic liver (S1/2/3) and normal/mild steatosis (S0/1) from moderate/severe steatosis (S2/3). Diagnostic performances were assessed with area under the receiver operating characteristic curves (AUC), sensitivity, specificity and balanced accuracy with 95% confidence interval (CI).

RESULTS

A total of 403 patients with 403 US exams were included: 171 (42%) were normal (S0: MRI-PDFF <5%), 154 (38%) had mild steatosis (S1: MRI-PDFF 5-17.4%), 29 (7%) had moderate steatosis (S2: MRI-PDFF >17.4%-22.1%) and 49 (12%) had severe steatosis (S3: MRI-PDFF >22.1%). The dataset was split to include 322 patients in train/validation and 81 patients in a holdout test set (kept blind). The S0 versus S1/2/3 model achieved 81.3% (95% CI 72.1-90.5) AUC, 81.1% (70.6-91.6) sensitivity, 71.4% (54.7-88.2) specificity and 76.3% (66.4-86.2) balanced accuracy. The S0/1 versus S2/3 model achieved 95.9% (89-100) AUC, 87.5% (71.3-100) sensitivity, 96.9% (92.7-100) specificity and 92.2% (83.8-100) balanced accuracy. A multi-class model achieved a sensitivity of 71.4% (54.7-88.2) for S0, 67.6% (52.5-82.7) for S1 and 87.5% (71.3-100) for S2/3; specificity for the same model was 81.1% (70.6-91.6) for S0, 77.3% (64.9-89.7) for S1 and 96.9% (92.7-100) for S2/3.

CONCLUSION

Our DL program offered high sensitivity and accuracy in detecting and categorizing hepatic steatosis from standard-of-care ultrasound.

EEG Signals Classification Related to Visual Objects Using Long Short-Term Memory Network and Nonlinear Interval Type-2 Fuzzy Regression

By gaining insights into how brain activity is encoded and decoded, we enhance our understanding of brain function. This study introduces a method for classifying EEG signals related to visual objects, employing a combination of an LSTM network and nonlinear interval type-2 fuzzy regression (NIT2FR). Here, ResNet is utilized for feature extraction from images, the LSTM network for feature extraction from EEG signals, and NIT2FR for mapping image features to EEG signal features. The application of type-2 fuzzy logic addresses uncertainties arising from EEG signal nonlinearity, noise, limited data sample size, and diverse mental states among participants. The Stanford database was used for implementation, evaluating effectiveness through metrics like classification accuracy, precision, recall, and F1 score. According to the findings, the LSTM network achieved an accuracy of 55.83% in categorizing images using raw EEG data. When compared to other methods like linear type-2, linear/nonlinear type-1 fuzzy, neural network, and polynomial regression, NIT2FR coupled with an SVM classifier outperformed with a 68.05% accuracy. Thus, NIT2FR demonstrates superiority in handling high uncertainty environments. Moreover, the 6.03% improvement in accuracy over the best previous study using the same dataset underscores its effectiveness. Precision, recall, and F1 score results for NIT2FR were 68.93%, 68.08%, and 68.49% respectively, surpassing outcomes from linear type-2, linear/nonlinear type-1 fuzzy regression methods.

Comparison of deep learning schemes in grading non-alcoholic fatty liver disease using B-mode ultrasound hepatorenal window images with liver biopsy as the gold standard

BACKGROUND/INTRODUCTION

To evaluate the performance of pre-trained deep learning schemes (DLS) in hepatic steatosis (HS) grading of Non-Alcoholic Fatty Liver Disease (NAFLD) patients, using as input B-mode US images containing right kidney (RK) cortex and liver parenchyma (LP) areas indicated by an expert radiologist.

METHODS

A total of 112 consecutively enrolled, biopsy-validated NAFLD patients underwent a regular abdominal B-mode US examination. For each patient, a radiologist obtained a B-mode US image containing RK cortex and LP and marked a point between the RK and LP, around which a window was automatically cropped. The cropped image dataset was augmented using up-sampling, and the augmented and non-augmented datasets were sorted by HS grade. Each dataset was split into training (70%) and testing (30%), and fed separately as input to InceptionV3, MobileNetV2, ResNet50, DenseNet201, and NASNetMobile pre-trained DLS. A receiver operating characteristic (ROC) analysis of hepatorenal index (HRI) measurements by the radiologist from the same cropped images was used for comparison with the performance of the DLS.

RESULTS

With the test data, the DLS reached 89.15 %-93.75 % accuracy when comparing HS grades S0-S1 vs. S2-S3 and 79.69 %-91.21 % accuracy for S0 vs. S1 vs. S2 vs. S3 with augmentation, and 80.45-82.73 % accuracy when comparing S0-S1 vs. S2-S3 and 59.54 %-63.64 % accuracy for S0 vs. S1 vs. S2 vs. S3 without augmentation. The performance of radiologists' HRI measurement after ROC analysis was 82 %, 91.56 %, and 96.19 % for thresholds of S ≥ S1, S ≥ S2, and S = S3, respectively.

CONCLUSION

All networks achieved high performance in HS assessment. DenseNet201 with the use of augmented data seems to be the most efficient supplementary tool for NAFLD diagnosis and grading.

Diagnostic Accuracy and Interobserver Reliability of Rotator Cuff Tear Detection With Ultrasonography Are Improved With Attentional Deep Learning

PURPOSE

To improve the accuracy of 1-stage object detection by modifying the YOLOv7 with the convolutional block attention module (CBAM), known as YOLOv7-CBAM, which can automatically identify torn or intact rotator cuff tendon to assist physicians in diagnosing rotator cuff lesions through ultrasound.

METHODS

Between 2020 and 2021, patients who experienced shoulder pain for over 3 months and had both ultrasound and magnetic resonance imaging examinations were categorized into torn and intact groups. To ensure balanced training, we included the same number of patients in both groups. Transfer learning was conducted using a pretrained model of YOLOv7 and an improved model with CBAM. The mean average precision, sensitivity, and F1-score were calculated to evaluate the models. A gradient-weighted class activation mapping method was employed to visualize important regions using a heatmap. A simulation data set was recruited to evaluate the diagnostic performance of clinical physicians using our artificial intelligence-assisted model.

RESULTS

A total of 280 patients were included in this study, with 80% of 840 ultrasound images randomly allocated for model training. The accuracy for the test set was 0.96 for YOLOv7 and 0.98 for YOLOv7-CBAM, and the precision and sensitivity were 0.94 and 0.98 for YOLOv7 and 0.98 and 0.98 for YOLOv7-CBAM. F1-score and mean average precision were higher for YOLOv7-CBAM (0.980 and 0.993) than YOLOv7 (0.961 and 0.965). Furthermore, the gradient-weighted class activation mapping method elucidated that the deep learning model primarily emphasized a hypoechoic anechoic defect within the tendon. Following adopting an artificial intelligence-assisted model (YOLOv7-CBAM model), diagnostic accuracy improved from 80.86% to 88.86% (P = .01), and interobserver reliability improved from 0.49 to 0.71 among physicians.

CONCLUSIONS

The YOLOv7-CBAM model shows high accuracy in detecting torn or intact rotator cuff tendon from ultrasound images. Integrating this model into the diagnostic process can assist physicians in improving diagnostic accuracy and interobserver reliability across different physicians.

CLINICAL RELEVANCE

The attentional deep learning model aids physicians in improving the accuracy and consistency of ultrasound diagnosis of torn or intact rotator cuff tendons.

Determining the sensitivity and specificity of the calculated fatty liver index in comparison with ultrasound

BACKGROUND

Non-alcoholic fatty liver disease (NAFLD) is a common chronic liver disease in human history and it is expected to surpass other causes of liver disease mortality by 2030. Therefore, finding an alternative way to diagnose steatosis in the early stage when imaging modalities are not available is crucial. This study decided to validate the optimal cut-off points and the sensitivity and specificity of the Fatty Liver Index (FLI) based on the Iranian population compared to ultrasonography.

METHODS

The data of 367 individuals, 108 males and 259 females over 35, were analyzed. Hepatic steatosis was identified by ultrasound. FLI was determined from waist circumference, gamma-glutamyl transferase, triglyceride, and body mass index data. The receiver operating characteristic curve (ROC) was used to determine the best FLI index cut point for diagnosing nonalcoholic fatty liver. The sensitivity and specificity indices were calculated for the determined cut point.

RESULTS

The AUC of the FLI index in diagnosing NAFLD in the total population was 0.733 (95% CI: 0.68-0.77, specificity = 0.6705, sensitivity = 0.7320) with the optimal COP of 40.6. There was a statistically significant association between non-alcoholic liver disease and FLI-based ultrasound (p < 0.0001). Furthermore, the sex-specific optimal COPs of FLI was 33.4, specificity = 0.6071, sensitivity = 0.8462 in men vs. 27.8, sensitivity = 0.8233, specificity = 0.7655 in women.

CONCLUSION

FLI is a reliable tool for identifying individuals with NAFLD. It has the potential to aid in detecting and managing this condition in large-scale populations while other methods are not available. We also determine an optimal COP of 40.6 with sensitivity and specificity of 73.20% and 67.05% in the general population, respectively.

A Multi-Institutional Study on Ultrasound Image Analysis for Staging HBV-Derived Liver Fibrosis: A Potential Noninvasive Alternative to Liver Stiffness Measurement

INTRODUCTION

Liver stiffness measurement is principal for staging liver fibrosis but not included in routine examinations. We investigated whether comparable diagnostic performance can be achieved by mining ultrasound images and developing a novel serum index (NSI).

METHODS

Texture features were extracted from ultrasound images. Spearman correlation and logistics regression selected independent variables for significant (F ≥ 2) and advanced (F ≥ 3) fibrosis. We compared the diagnostic performance of transient elastography (TE), ultrasound image biomarker, conventional serum indices (aspartate aminotransferase-to-platelet ratio index, fibrosis-4 index, gamma-glutamyl transpeptidase-to-platelet ratio), and NSI in 365 patients with chronic hepatitis B.

RESULTS

Among patients, 52.1% had significant fibrosis and 24.2% had advanced fibrosis. PLT, gamma-glutamyl transferase, prealbumin, and globulin were incorporated into NSI. In the validation group, TE achieved the best performance (area under the curve [AUC]: 0.765 [0.690-0.849] for significant fibrosis; 0.812 [0.745-0.878] for advanced fibrosis), followed by ultrasound image biomarker (AUC: 0.712 [0.629-0.795]; 0.678 [0.595-0.763]) and NSI (AUC: 0.630 [0.534-0.725]; 0.659 [0.572-0.745]), outperforming conventional indices.

DISCUSSION

Texture analysis enhances ultrasound's diagnostic utility, but TE remains superior. When TE is unavailable, ultrasound image analysis and NSI, incorporating prealbumin, can serve as alternative tools for fibrosis staging.

Machine Learning Models for Predicting Significant Liver Fibrosis in Patients with Severe Obesity and Nonalcoholic Fatty Liver Disease

PURPOSE

Although noninvasive tests can be used to predict liver fibrosis, their accuracy is limited for patients with severe obesity and nonalcoholic fatty liver disease (NAFLD). We developed machine learning (ML) models to predict significant liver fibrosis in patients with severe obesity through noninvasive tests.

MATERIALS AND METHODS

This prospective study included 194 patients with severe obesity who underwent wedge liver biopsy and metabolic bariatric surgery at Taipei Medical University Hospital between September 2016 and December 2020. Significant liver fibrosis was defined as a fibrosis score ≥ 2. Patients were randomly divided into a training group (70%) and a validation group (30%). ML models, including support vector machine, random forest, k-nearest neighbor, XGBoost, and logistic regression, were trained to predict significant liver fibrosis, using DM status, AST, ALT, ultrasonographic fibrosis scores, and liver stiffness measurements (LSM). An ensemble model including these ML models was also used for prediction.

RESULTS

Among the ML models, the XGBoost model exhibited the highest AUROC of 0.77, with a sensitivity, specificity, and accuracy of 61.5%, 75.8%, and 69.5%, in validation set, while LSM, AST, ALT showed strongest effects on the model. The ensemble model outperformed all ML models in terms of sensitivity, specificity, and accuracy of 73.1%, 90.9%, and 83.1%.

CONCLUSION

For patients with severe obesity and NAFLD, the XGBoost model and the ensemble model exhibit high predictive performance for significant liver fibrosis. These models may be used to screen for significant liver fibrosis in this patient group and monitor treatment response after metabolic bariatric surgery.

ANN uncertainty estimates in assessing fatty liver content from ultrasound data

Background and objective

This article uses three different probabilistic convolutional architectures applied to ultrasound image analysis for grading Fatty Liver Content (FLC) in Metabolic Dysfunction Associated Steatotic Liver Disease (MASLD) patients. Steatosis is a new silent epidemic and its accurate measurement is an impelling clinical need, not only for hepatologists, but also for experts in metabolic and cardiovascular diseases. This paper aims to provide a robust comparison between different uncertainty quantification strategies to identify advantages and drawbacks in a real clinical setting.

Methods

We used a classical Convolutional Neural Network, a Monte Carlo Dropout, and a Bayesian Convolutional Neural Network with the goal of not only comparing the goodness of the predictions, but also to have access to an evaluation of the uncertainty associated with the outputs.

Results

We found that even if the prediction based on a single ultrasound view is reliable (relative RMSE [5.93%-12.04%]), networks based on two ultrasound views outperform them (relative RMSE [5.35%-5.87%]). In addition, the results show that the introduction of a "not confident" category contributes to increase the percentage of correctly predicted cases and to decrease the percentage of mispredicted cases, especially for semi-intrusive methods.

Conclusions

The possibility of having access to information about the confidence with which the network produces its outputs is a great advantage, both from the point of view of physicians who want to use neural networks as computer-aided diagnosis, and for developers who want to limit overfitting and obtain information about dataset problems in terms of out-of-distribution detection.

Ultrasound Versus Elastography in the Diagnosis of Hepatic Steatosis: Evaluation of Traditional Machine Learning Versus Deep Learning

The prevalence of fatty liver disease is on the rise, posing a significant global health concern. If left untreated, it can progress into more serious liver diseases. Therefore, accurately diagnosing the condition at an early stage is essential for more effective intervention and management. This study uses images acquired via ultrasound and elastography to classify liver steatosis using classical machine learning classifiers, including random forest and support vector machine, as well as deep learning architectures, such as ResNet50V2 and DenseNet-201. The neural network demonstrated the most optimal performance, achieving an F1 score of 99.5% on the ultrasound dataset, 99.2% on the elastography dataset, and 98.9% on the mixed dataset. The results from the deep learning approach are comparable to those of machine learning, despite objectively not achieving the highest results. This research offers valuable insights into the domain of medical image classification and advocates the integration of advanced machine learning and deep learning technologies in diagnosing steatosis.

Non-invasive diagnosis of pancreatic steatosis with ultrasound images using deep learning network

Objective

This study aimed to verify whether pancreatic steatosis (PS) is an independent risk factor for type 2 diabetes mellitus (T2DM). We also developed and validated a deep learning model for the diagnosis of PS using ultrasonography (US) images based on histological classifications.

Methods

In this retrospective study, we analysed data from 139 patients who underwent US imaging of the pancreas followed by pancreatic resection at our medical institution. Logistic regression analysis was employed to ascertain the independent predictors of T2DM. The diagnostic efficacy of the deep learning model for PS was assessed using receiver operating characteristic curve analysis and compared with traditional visual assessment methodology in US imaging.

Results

The incidence rate of PS in the study cohort was 64.7 %. Logistic regression analysis revealed that age (P = 0.003) and the presence of PS (P = 0.048) were independent factors associated with T2DM. The deep learning model demonstrated robust diagnostic capabilities for PS, with areas under the curve of 0.901 and 0.837, sensitivities of 0.895 and 0.920, specificities of 0.700 and 0.765, accuracies of 0.814 and 0.857, and F1-scores of 0.850 and 0.885 for the training and validation cohorts, respectively. These metrics significantly outperformed those of conventional US imaging (P < 0.001 and P = 0.045, respectively).

Conclusion

The deep learning model significantly enhanced the diagnostic accuracy of conventional ultrasound for PS detection. Its high sensitivity could facilitate widespread screening for PS in large populations, aiding in the early identification of individuals at an elevated risk for T2DM in routine clinical practice.

Laboratory variables-based artificial neural network models for predicting fatty liver disease: A retrospective study

Background

The efficacy of artificial neural network (ANN) models employing laboratory variables for predicting fatty liver disease (FLD) remains inadequately established. The study aimed to develop ANN models to precisely predict FLD.

Methods

Of 12,058 participants undergoing the initial FLD screening, 7,990 eligible participants were included. A total of 6,309 participants were divided randomly into the training (4,415 participants, 70%) and validation (1,894 participants, 30%) sets for developing prediction models. The performance of ANNs was additionally tested in the testing set (1,681 participants). The area under the receiver operating characteristic curve (AUROC) was employed to assess the models' performance.

Results

The 18-variable, 11-variable, 3-variable, and 2-variable models each achieved robust FLD prediction performance, with AUROCs over 0.92, 0.91, and 0.89 in the training, validation, and testing, respectively. Although slightly inferior to the other three models in performance (AUROC ranges: 0.89-0.92 vs 0.91-0.95), the 2-variable model showed 80.3% accuracy and 89.7% positive predictive value in the testing. Incorporating age and gender increased the AUROCs of the resulting 20-variable, 13-variable, 5-variable, and 4-variable models each to over 0.93, 0.92, and 0.91 in the training, validation, and testing, respectively.

Conclusions

Implementation of the ANN models could effectively predict FLD, with enhanced predictive performance via the inclusion of age and gender.

Neural Network Classification Algorithm Based on Self-attention Mechanism and Ensemble Learning for MASLD Ultrasound Images

BACKGROUND

Ultrasound image examination has become the preferred choice for diagnosing metabolic dysfunction-associated steatotic liver disease (MASLD) due to its non-invasive nature. Computer-aided diagnosis (CAD) technology can assist doctors in avoiding deviations in the detection and classification of MASLD.

METHOD

We propose a hybrid model that integrates the pre-trained VGG16 network with an attention mechanism and a stacking ensemble learning model, which is capable of multi-scale feature aggregation based on the self-attention mechanism and multi-classification model fusion (Logistic regression, random forest, support vector machine) based on stacking ensemble learning. The proposed hybrid method achieves four classifications of normal, mild, moderate, and severe fatty liver based on ultrasound images.

RESULT AND CONCLUSION

Our proposed hybrid model reaches an accuracy of 91.34% and exhibits superior robustness against interference, which is better than traditional neural network algorithms. Experimental results show that, compared with the pre-trained VGG16 model, adding the self-attention mechanism improves the accuracy by 3.02%. Using the stacking ensemble learning model as a classifier further increases the accuracy to 91.34%, exceeding any single classifier such as LR (89.86%) and SVM (90.34%) and RF (90.73%). The proposed hybrid method can effectively improve the efficiency and accuracy of MASLD ultrasound image detection.

Early and accurate diagnosis of steatotic liver by artificial intelligence (AI)-supported ultrasonography

OBJECTIVES

Steatotic liver disease is the most frequent chronic liver disease worldwide. Ultrasonography (US) is commonly employed for the assessment and diagnosis. Few information is available on the possible use of artificial intelligence (AI) to ameliorate the diagnostic accuracy of ultrasonography.

MATERIALS AND METHODS

An AI-based algorithm was developed using a dataset of US images. We prospectively enrolled 134 patients for algorithm validation. Patients underwent abdominal US and Proton Density Fat Fraction MRI scans (MRI-PDFF), assumed as reference technique. The hepatorenal index was manually calculated (HRIM) by 4 operators. An automatic hepatorenal index (HRIA) was obtained by the algorithm. The accuracy of HRIA to discriminate steatosis grades was evaluated by ROC analysis using MRI-PDFF cut-offs.

RESULTS

Overweight was 40 % of subjects (BMI 26.4 kg/cm2). The median HRIA was 1.11 (IQR 0.32) and the average of 4 manually calculated HRIM was 1.08 (IQR 0.26), with a 15 % inter-operator variability. Both HRIA (R = 0.79, P < 0.0001) and HRIM (R = 0.69, P < 0.0001) significantly correlated with liver fat percentage (MRI-PDFF). According to MRI-PDFF, 32 % of enrolled subjects had steatosis. Discrimination capacity by AUC between patient with steatosis and patient without steatosis was better for HRIA than HRIM (AUC: 0.87 vs. 0.82, respectively). ROC analysis showed an AUC = 0.98 for HRIA with 1.64 cut-off in distinguishing between mild and moderate/severe groups.

CONCLUSIONS

The use of AI improves accuracy and speed of ultrasonography in the diagnosis of liver steatosis. Further studies should evaluate the routine use of this technique in the management of liver steatosis at high cardio-metabolic risk.

Multiclassification of Hepatic Cystic Echinococcosis by Using Multiple Kernel Learning Framework and Ultrasound Images

To properly treat and care for hepatic cystic echinococcosis (HCE), it is essential to make an accurate diagnosis before treatment.

OBJECTIVE

The objective of this study was to assess the diagnostic accuracy of computer-aided diagnosis techniques in classifying HCE ultrasound images into five subtypes.

METHODS

A total of 1820 HCE ultrasound images collected from 967 patients were included in the study. A multi-kernel learning method was developed to learn the texture and depth features of the ultrasound images. Combined kernel functions were built-in Support Vector Machine (MK-SVM) for the classification work. The experimental results were evaluated using five-fold cross-validation. Finally, our approach was compared with three other machine learning algorithms: the decision tree classifier, random forest, and gradient boosting decision tree.

RESULTS

Among all the methods used in the study, the MK-SVM achieved the highest accuracy of 96.6% on the fused feature set.

CONCLUSION

The multi-kernel learning method effectively learns different image features from ultrasound images by utilizing various kernels. The MK-SVM method, which combines the learning of texture features and depth features separately, has significant application value in HCE classification tasks.

Simulating federated learning for steatosis detection using ultrasound images

We aimed to implement four data partitioning strategies evaluated with four federated learning (FL) algorithms and investigate the impact of data distribution on FL model performance in detecting steatosis using B-mode US images. A private dataset (153 patients; 1530 images) and a public dataset (55 patient; 550 images) were included in this retrospective study. The datasets contained patients with metabolic dysfunction-associated fatty liver disease (MAFLD) with biopsy-proven steatosis grades and control individuals without steatosis. We employed four data partitioning strategies to simulate FL scenarios and we assessed four FL algorithms. We investigated the impact of class imbalance and the mismatch between the global and local data distributions on the learning outcome. Classification performance was assessed with area under the receiver operating characteristic curve (AUC) on a separate test set. AUCs were 0.93 (95% CI 0.92, 0.94) for source-based partitioning scenario with FedAvg, 0.90 (95% CI 0.89, 0.91) for a centralized model, and 0.83 (95% CI 0.81, 0.85) for a model trained in a single-center scenario. When data was perfectly balanced on the global level and each site had an identical data distribution, the model yielded an AUC of 0.90 (95% CI 0.88, 0.92). When each site contained data exclusively from one single class, irrespective of the global data distribution, the AUC fell in the range of 0.34-0.70. FL applied to B-mode US images provide performance comparable to a centralized model and higher than single-center scenario. Global data imbalance and local data heterogeneity influenced the learning outcome.

Real-Time Precise Targeting of the Subthalamic Nucleus via Transfer Learning in a Rat Model of Parkinson's Disease Based on Microelectrode Arrays

In neurodegenerative disorders, neuronal firing patterns and oscillatory activity are remarkably altered in specific brain regions, which can serve as valuable biomarkers for the identification of deep brain regions. The subthalamic nucleus (STN) has been the primary target for DBS in patients with Parkinson's disease (PD). In this study, changes in the spike firing patterns and spectral power of local field potentials (LFPs) in the pre-STN (zona incerta, ZI) and post-STN (cerebral peduncle, cp) regions were investigated in PD rats, providing crucial evidence for the functional localization of the STN. Sixteen-channel microelectrode arrays (MEAs) with sites distributed at different depths and widths were utilized to record neuronal activities. The spikes in the STN exhibited higher firing rates than those in the ZI and cp. Furthermore, the LFP power in the delta band in the STN was the greatest, followed by that in the ZI, and was greater than that in the cp. Additionally, increased LFP power was observed in the beta bands in the STN. To identify the best performing classification model, we applied various convolutional neural networks (CNNs) based on transfer learning to analyze the recorded raw data, which were processed using the Gram matrix of the spikes and the fast Fourier transform of the LFPs. The best transfer learning model achieved an accuracy of 95.16%. After fusing the spike and LFP classification results, the time precision for processing the raw data reached 500 ms. The pretrained model, utilizing raw data, demonstrated the feasibility of employing transfer learning for training models on neural activity. This approach highlights the potential for functional localization within deep brain regions.

Fatty liver classification via risk controlled neural networks trained on grouped ultrasound image data

Ultrasound imaging is a widely used technique for fatty liver diagnosis as it is practically affordable and can be quickly deployed by using suitable devices. When it is applied to a patient, multiple images of the targeted tissues are produced. We propose a machine learning model for fatty liver diagnosis from multiple ultrasound images. The machine learning model extracts features of the ultrasound images by using a pre-trained image encoder. It further produces a summary embedding on these features by using a graph neural network. The summary embedding is used as input for a classifier on fatty liver diagnosis. We train the machine learning model on a ultrasound image dataset collected by Taiwan Biobank. We also carry out risk control on the machine learning model using conformal prediction. Under the risk control procedure, the classifier can improve the results with high probabilistic guarantees.

Artificial intelligence: Applications in critical care gastroenterology

Gastrointestinal (GI) complications frequently necessitate intensive care unit (ICU) admission. Additionally, critically ill patients also develop GI complications requiring further diagnostic and therapeutic interventions. However, these patients form a vulnerable group, who are at risk for developing side effects and complications. Every effort must be made to reduce invasiveness and ensure safety of interventions in ICU patients. Artificial intelligence (AI) is a rapidly evolving technology with several potential applications in healthcare settings. ICUs produce a large amount of data, which may be employed for creation of AI algorithms, and provide a lucrative opportunity for application of AI. However, the current role of AI in these patients remains limited due to lack of large-scale trials comparing the efficacy of AI with the accepted standards of care.

Artificial Intelligence in Liver Diseases: Recent Advances

Liver diseases cause a significant burden on public health worldwide. In spite of great advances during recent years, there are still many challenges in the diagnosis and treatment of liver diseases. During recent years, artificial intelligence (AI) has been widely used for the diagnosis, risk stratification, and prognostic prediction of various diseases based on clinical datasets and medical images. Accumulative studies have shown its performance for diagnosing patients with nonalcoholic fatty liver disease and liver fibrosis and assessing their severity, and for predicting treatment response and recurrence of hepatocellular carcinoma, outcomes of liver transplantation recipients, and risk of drug-induced liver injury. Herein, we aim to comprehensively summarize the current evidence regarding diagnostic, prognostic, and/or therapeutic role of AI in these common liver diseases.

Application of artificial intelligence techniques for non-alcoholic fatty liver disease diagnosis: A systematic review (2005-2023)

BACKGROUND AND OBJECTIVES

Non-alcoholic fatty liver disease (NAFLD) is a common liver disease with a rapidly growing incidence worldwide. For prognostication and therapeutic decisions, it is important to distinguish the pathological stages of NAFLD: steatosis, steatohepatitis, and liver fibrosis, which are definitively diagnosed on invasive biopsy. Non-invasive ultrasound (US) imaging, including US elastography technique, and clinical parameters can be used to diagnose and grade NAFLD and its complications. Artificial intelligence (AI) is increasingly being harnessed for developing NAFLD diagnostic models based on clinical, biomarker, or imaging data. In this work, we systemically reviewed the literature for AI-enabled NAFLD diagnostic models based on US (including elastography) and clinical (including serological) data.

METHODS

We performed a comprehensive search on Google Scholar, Scopus, and PubMed search engines for articles published between January 2005 and June 2023 related to AI models for NAFLD diagnosis based on US and/or clinical parameters using the following search terms: "non-alcoholic fatty liver disease", "non-alcoholic steatohepatitis", "deep learning", "machine learning", "artificial intelligence", "ultrasound imaging", "sonography", "clinical information".

RESULTS

We reviewed 64 published models that used either US (including elastography) or clinical data input to detect the presence of NAFLD, non-alcoholic steatohepatitis, and/or fibrosis, and in some cases, the severity of steatosis, inflammation, and/or fibrosis as well. The performances of the published models were summarized, and stratified by data input and algorithms used, which could be broadly divided into machine and deep learning approaches.

CONCLUSION

AI models based on US imaging and clinical data can reliably detect NAFLD and its complications, thereby reducing diagnostic costs and the need for invasive liver biopsy. The models offer advantages of efficiency, accuracy, and accessibility, and serve as virtual assistants for specialists to accelerate disease diagnosis and reduce treatment costs for patients and healthcare systems.

Diagnosis of Salivary Gland Tumors Using Transfer Learning with Fine-Tuning and Gradual Unfreezing

Ultrasound is the primary tool for evaluating salivary gland tumors (SGTs); however, tumor diagnosis currently relies on subjective features. This study aimed to establish an objective ultrasound diagnostic method using deep learning. We collected 446 benign and 223 malignant SGT ultrasound images in the training/validation set and 119 benign and 44 malignant SGT ultrasound images in the testing set. We trained convolutional neural network (CNN) models from scratch and employed transfer learning (TL) with fine-tuning and gradual unfreezing to classify malignant and benign SGTs. The diagnostic performances of these models were compared. By utilizing the pretrained ResNet50V2 with fine-tuning and gradual unfreezing, we achieved a 5-fold average validation accuracy of 0.920. The diagnostic performance on the testing set demonstrated an accuracy of 89.0%, a sensitivity of 81.8%, a specificity of 91.6%, a positive predictive value of 78.3%, and a negative predictive value of 93.2%. This performance surpasses that of other models in our study. The corresponding Grad-CAM visualizations were also presented to provide explanations for the diagnosis. This study presents an effective and objective ultrasound method for distinguishing between malignant and benign SGTs, which could assist in preoperative evaluation.

Comparison of Radiologists and Deep Learning for US Grading of Hepatic Steatosis

Background Screening for nonalcoholic fatty liver disease (NAFLD) is suboptimal due to the subjective interpretation of US images. Purpose To evaluate the agreement and diagnostic performance of radiologists and a deep learning model in grading hepatic steatosis in NAFLD at US, with biopsy as the reference standard. Materials and Methods This retrospective study included patients with NAFLD and control patients without hepatic steatosis who underwent abdominal US and contemporaneous liver biopsy from September 2010 to October 2019. Six readers visually graded steatosis on US images twice, 2 weeks apart. Reader agreement was assessed with use of κ statistics. Three deep learning techniques applied to B-mode US images were used to classify dichotomized steatosis grades. Classification performance of human radiologists and the deep learning model for dichotomized steatosis grades (S0, S1, S2, and S3) was assessed with area under the receiver operating characteristic curve (AUC) on a separate test set. Results The study included 199 patients (mean age, 53 years ± 13 [SD]; 101 men). On the test set (n = 52), radiologists had fair interreader agreement (0.34 [95% CI: 0.31, 0.37]) for classifying steatosis grades S0 versus S1 or higher, while AUCs were between 0.49 and 0.84 for radiologists and 0.85 (95% CI: 0.83, 0.87) for the deep learning model. For S0 or S1 versus S2 or S3, radiologists had fair interreader agreement (0.30 [95% CI: 0.27, 0.33]), while AUCs were between 0.57 and 0.76 for radiologists and 0.73 (95% CI: 0.71, 0.75) for the deep learning model. For S2 or lower versus S3, radiologists had fair interreader agreement (0.37 [95% CI: 0.33, 0.40]), while AUCs were between 0.52 and 0.81 for radiologists and 0.67 (95% CI: 0.64, 0.69) for the deep learning model. Conclusion Deep learning approaches applied to B-mode US images provided comparable performance with human readers for detection and grading of hepatic steatosis. Published under a CC BY 4.0 license. Supplemental material is available for this article. See also the editorial by Tuthill in this issue.

Artificial Intelligence (AI) Game Changer in Cancer Biology

Healthcare is one of many industries where the most modern technologies, such as artificial intelligence and machine learning, have shown a wide range of applications. Cancer, one of the most prevalent non-communicable diseases in modern times, accounts for a sizable portion of worldwide mortality. Investigations are continuously being conducted to find ways to reduce cancer mortality and morbidity. Artificial Intelligence (AI) is currently being used in cancer research, with promising results. Two main features play a vital role in improving cancer prognosis: early detection and proper diagnosis using imaging and molecular techniques. AI's use as a tool in these sectors has demonstrated its capacity to precisely detect and diagnose, which is one of AI's many applications in cancer research. The purpose of this chapter is to review the literature and find AI applications in a range of cancers that are commonly seen.

Nonalcoholic fatty liver disease (NAFLD) detection and deep learning in a Chinese community-based population

OBJECTIVES

We aimed to develop and validate a deep learning system (DLS) by using an auxiliary section that extracts and outputs specific ultrasound diagnostic features to improve the explainable, clinical relevant utility of using DLS for detecting NAFLD.

METHODS

In a community-based study of 4144 participants with abdominal ultrasound scan in Hangzhou, China, we sampled 928 (617 [66.5%] females, mean age: 56 years ± 13 [standard deviation]) participants (2 images per participant) to develop and validate DLS, a two-section neural network (2S-NNet). Radiologists' consensus diagnosis classified hepatic steatosis as none steatosis, mild, moderate, and severe. We also explored the NAFLD detection performance of six one-section neural network models and five fatty liver indices on our data set. We further evaluated the influence of participants' characteristics on the correctness of 2S-NNet by logistic regression.

RESULTS

Area under the curve (AUROC) of 2S-NNet for hepatic steatosis was 0.90 for ≥ mild, 0.85 for ≥ moderate, and 0.93 for severe steatosis, and was 0.90 for NAFLD presence, 0.84 for moderate to severe NAFLD, and 0.93 for severe NAFLD. The AUROC of NAFLD severity was 0.88 for 2S-NNet, and 0.79-0.86 for one-section models. The AUROC of NAFLD presence was 0.90 for 2S-NNet, and 0.54-0.82 for fatty liver indices. Age, sex, body mass index, diabetes, fibrosis-4 index, android fat ratio, and skeletal muscle via dual-energy X-ray absorptiometry had no significant impact on the correctness of 2S-NNet (p > 0.05).

CONCLUSIONS

By using two-section design, 2S-NNet had improved the performance for detecting NAFLD with more explainable, clinical relevant utility than using one-section design.

KEY POINTS

• Based on the consensus review derived from radiologists, our DLS (2S-NNet) had an AUROC of 0.88 by using two-section design and yielded better performance for detecting NAFLD than using one-section design with more explainable, clinical relevant utility. • The 2S-NNet outperformed five fatty liver indices with the highest AUROCs (0.84-0.93 vs. 0.54-0.82) for different NAFLD severity screening, indicating screening utility of deep learning-based radiology may perform better than blood biomarker panels in epidemiology. • The correctness of 2S-NNet was not significantly influenced by individual's characteristics, including age, sex, body mass index, diabetes, fibrosis-4 index, android fat ratio, and skeletal muscle via dual-energy X-ray absorptiometry.

Radiological Diagnosis of Chronic Liver Disease and Hepatocellular Carcinoma: A Review

Medical image analysis plays a pivotal role in the evaluation of diseases, including screening, surveillance, diagnosis, and prognosis. Liver is one of the major organs responsible for key functions of metabolism, protein and hormone synthesis, detoxification, and waste excretion. Patients with advanced liver disease and Hepatocellular Carcinoma (HCC) are often asymptomatic in the early stages; however delays in diagnosis and treatment can lead to increased rates of decompensated liver diseases, late-stage HCC, morbidity and mortality. Ultrasound (US) is commonly used imaging modality for diagnosis of chronic liver diseases that includes fibrosis, cirrhosis and portal hypertension. In this paper, we first provide an overview of various diagnostic methods for stages of liver diseases and discuss the role of Computer-Aided Diagnosis (CAD) systems in diagnosing liver diseases. Second, we review the utility of machine learning and deep learning approaches as diagnostic tools. Finally, we present the limitations of existing studies and outline future directions to further improve diagnostic accuracy, as well as reduce cost and subjectivity, while also improving workflow for the clinicians.

Bio-Inspired Network for Diagnosing Liver Steatosis in Ultrasound Images

Using ultrasound imaging to diagnose liver steatosis is of great significance for preventing diseases such as cirrhosis and liver cancer. Accurate diagnosis under conditions of low quality, noise and poor resolutions is still a challenging task. Physiological studies have shown that the visual cortex of the biological visual system has selective attention neural mechanisms and feedback regulation of high features to low features. When processing visual information, these cortical regions selectively focus on more sensitive information and ignore unimportant details, which can effectively extract important features from visual information. Inspired by this, we propose a new diagnostic network for hepatic steatosis. In order to simulate the selection mechanism and feedback regulation of the visual cortex in the ventral pathway, it consists of a receptive field feature extraction module, parallel attention module and feedback connection. The receptive field feature extraction module corresponds to the inhibition of the non-classical receptive field of V1 neurons on the classical receptive field. It processes the input image to suppress the unimportant background texture. Two types of attention are adopted in the parallel attention module to process the same visual information and extract different important features for fusion, which improves the overall performance of the model. In addition, we construct a new dataset of fatty liver ultrasound images and validate the proposed model on this dataset. The experimental results show that the network has good performance in terms of sensitivity, specificity and accuracy for the diagnosis of fatty liver disease.

Radiobioinformatics: A novel bridge between basic research and clinical practice for clinical decision support in diffuse liver diseases

The liver is a multifaceted organ that is responsible for many critical functions encompassing amino acid, carbohydrate, and lipid metabolism, all of which make a healthy liver essential for the human body. Contemporary imaging methodologies have remarkable diagnostic accuracy in discerning focal liver lesions; however, a comprehensive understanding of diffuse liver diseases is a requisite for radiologists to accurately diagnose or predict the progression of such lesions within clinical contexts. Nonetheless, the conventional attributes of radiological features, including morphology, size, margin, density, signal intensity, and echoes, limit their clinical utility. Radiomics is a widely used approach that is characterized by the extraction of copious image features from radiographic depictions, which gives it considerable potential in addressing this limitation. It is worth noting that functional or molecular alterations occur significantly prior to the morphological shifts discernible by imaging modalities. Consequently, the explication of potential mechanisms by multiomics analyses (encompassing genomics, epigenomics, transcriptomics, proteomics, and metabolomics) is essential for investigating putative signal pathway regulations from a radiological viewpoint. In this review, we elaborate on the principal pathological categorizations of diffuse liver diseases, the evaluation of multiomics approaches pertaining to diffuse liver diseases, and the prospective value of predictive models. Accordingly, the overarching objective of this review is to scrutinize the interrelations between radiological features and bioinformatics as well as to consider the development of prediction models predicated on radiobioinformatics as integral components of clinical decision support systems for diffuse liver diseases.

A Paradigm Shift in Primary Liver Cancer Therapy Utilizing Genomics, Molecular Biomarkers, and Artificial Intelligence

Primary liver cancer is the sixth most common cancer worldwide and the third leading cause of cancer-related death. Conventional therapies offer limited survival benefit despite improvements in locoregional liver-directed therapies, which highlights the underlying complexity of liver cancers. This review explores the latest research in primary liver cancer therapies, focusing on developments in genomics, molecular biomarkers, and artificial intelligence. Attention is also given to ongoing research and future directions of immunotherapy and locoregional therapies of primary liver cancers.

Aggregated micropatch-based deep learning neural network for ultrasonic diagnosis of cirrhosis

Despite the advancements in the diagnosis of early-stage cirrhosis, the accuracy in the diagnosis using ultrasound is still challenging owing to the presence of various image artifacts, which results in poor visual quality of the textural and lower-frequency components. In this study, we propose an end-to-end multistep network called CirrhosisNet that includes two transfer-learned convolutional neural networks for semantic segmentation and classification tasks. It uses a uniquely designed image, called an aggregated micropatch (AMP), as an input image to the classification network, thereby assessing whether the liver is in a cirrhotic stage. With a prototype AMP image, we synthesized a bunch of AMP images while retaining the textural features. This synthesis significantly increases the number of insufficient cirrhosis-labeled images, thereby circumventing overfitting issues and optimizing network performance. Furthermore, the synthesized AMP images contained unique textural patterns, mostly generated on the boundaries between adjacent micropatches (μ-patches) during their aggregation. These newly created boundary patterns provide rich information regarding the texture features of the ultrasound image, thereby making cirrhosis diagnosis more accurate and sensitive. The experimental results demonstrated that our proposed AMP image synthesis is extremely effective in expanding the dataset of cirrhosis images, thus diagnosing liver cirrhosis with considerably high accuracy. We achieved an accuracy of 99.95 %, a sensitivity of 100 %, and a specificity of 99.9 % on the Samsung Medical Center dataset using 8 × 8 pixels-sized μ-patches. The proposed approach provides an effective solution to deep-learning models with limited-training data, such as medical imaging tasks.

Automated fatty liver disease detection in point-of-care ultrasound B-mode images

Purpose

Non-alcoholic fatty liver disease (NAFLD) is an increasing global health concern, with a prevalence of 25% worldwide. The rising incidence of NAFLD, an asymptomatic condition, reinforces the need for systematic screening strategies in primary care. We present the use of non-expert acquired point-of-care ultrasound (POCUS) B-mode images for the development of an automated steatosis classification algorithm.

Approach

We obtained a Health Insurance Portability and Accountability Act compliant dataset consisting of 478 patients [body mass index 23.60 ± 3.55 , age 40.97 ± 10.61 ], imaged with POCUS by non-expert health care personnel. A U-Net deep learning (DL) model was used for liver segmentation in the POCUS B-mode images, followed by 224 × 224 patch extraction of liver parenchyma. Several DL models including VGG-16, ResNet-50, Inception V3, and DenseNet-121 were trained for binary classification of steatosis. All layers of each tested model were unfrozen, and the final layer was replaced with a custom classifier. Majority voting was applied for patient-level results.

Results

On a hold-out test set of 81 patients, the final DenseNet-121 model yielded an area under the receiver operator characteristic curve of 90.1%, sensitivity of 95.0%, and specificity of 85.2% for the detection of liver steatosis. Average cross-validation performance in models using patches of liver parenchyma as input outperformed methods using complete B-mode frames.

Conclusions

Despite minimal POCUS acquisition training, and low-quality B-mode images, it is possible to detect steatosis using DL algorithms. Implementation of this algorithm in POCUS software may offer an accessible, low-cost steatosis screening technology, for use by non-expert health care personnel.

Artificial Intelligence (AI)-Enhanced Ultrasound Techniques Used in Non-Alcoholic Fatty Liver Disease: Are They Ready for Prime Time?

Non-alcoholic fatty liver disease (NAFLD) is the most prevalent cause of chronic liver disease, affecting approximately 2 billion individuals worldwide with a spectrum that can range from simple steatosis to cirrhosis. Typically, the diagnosis of NAFLD is based on imaging studies, but the gold standard remains liver biopsies. Hence, the use of artificial intelligence (AI) in this field, which has recently undergone rapid development in various aspects of medicine, has the potential to accurately diagnose NAFLD and steatohepatitis (NASH). This paper provides an overview of the latest research that employs AI for the diagnosis and staging of NAFLD, as well as applications for future developments in this field.

Noninvasive Staging of Hepatic Steatosis Using Calibrated 2D US with Liver Biopsy as the Reference Standard

Background Accumulation of lipid in the liver (ie, hepatic steatosis) is the basis of nonalcoholic fatty liver disease (NAFLD). Asymptomatic steatosis can lead to nonalcoholic steatohepatitis and downstream complications. Purpose To assess the diagnostic performance of calibrated US (CAUS) as a method for detection and staging of hepatic steatosis in comparison with liver biopsy. Materials and Methods Two-dimensional US images in 223 consecutive patients who underwent US-guided liver biopsy from May 2012 to February 2016 were retrospectively analyzed by two observers using CAUS. CAUS semiautomatically estimates echo-level and texture parameters, with particular interest in the residual attenuation coefficient (RAC), which is the remaining steatosis-driven attenuation obtained after correction of the beam profile. Data were correlated with patient characteristics and histologically determined steatosis grades and fibrosis stages. The data were equally divided into training and test sets to independently train and test logistic regression models for detection (>5% fat) and staging (>33% and >66% fat) of hepatic steatosis by using area under the receiver operating characteristic curve (AUC) analysis. Results A total of 195 patients (mean age, 50 years ± 13 [SD]; 110 men) were included and divided into a training set (n = 97 [50%]) and a test set (n = 98 [50%]). The average CAUS interobserver correlation coefficient was 0.95 (R range, 0.87-0.99). The best correlation with steatosis was found for the RAC parameter (R = 0.78, P < .01), while no correlation was found for fibrosis (R = 0.14, P = .054). Steatosis detection using RAC showed an AUC of 0.97 (95% CI: 0.94, 1.00), and the multivariable AUC was found to be 0.97 (95% CI: 0.95, 1.00). The predictive performance for moderate and severe hepatic steatosis using RAC was 0.93 (95% CI: 0.88, 0.98) and 0.93 (95% CI: 0.87, 0.98), respectively. Conclusion The calibrated US parameter residual attenuation coefficient detects and stages steatosis accurately with limited interobserver variability, and performance is not hampered by the presence of fibrosis. © RSNA, 2022 Online supplemental material is available for this article. See also the editorial by Grant in this issue.

Pre-training in Medical Data: A Survey

Medical data refers to health-related information associated with regular patient care or as part of a clinical trial program. There are many categories of such data, such as clinical imaging data, bio-signal data, electronic health records (EHR), and multi-modality medical data. With the development of deep neural networks in the last decade, the emerging pre-training paradigm has become dominant in that it has significantly improved machine learning methods’ performance in a data-limited scenario. In recent years, studies of pre-training in the medical domain have achieved significant progress. To summarize these technology advancements, this work provides a comprehensive survey of recent advances for pre-training on several major types of medical data. In this survey, we summarize a large number of related publications and the existing benchmarking in the medical domain. Especially, the survey briefly describes how some pre-training methods are applied to or developed for medical data. From a data-driven perspective, we examine the extensive use of pre-training in many medical scenarios. Moreover, based on the summary of recent pre-training studies, we identify several challenges in this field to provide insights for future studies.

Ultrasound-Based Machine Learning Approach for Detection of Nonalcoholic Fatty Liver Disease

OBJECTIVES

Current diagnosis of nonalcoholic fatty liver disease (NAFLD) relies on biopsy or MR-based fat quantification. This prospective study explored the use of ultrasound with artificial intelligence for the detection of NAFLD.

METHODS

One hundred and twenty subjects with clinical suspicion of NAFLD and 10 healthy volunteers consented to participate in this institutional review board-approved study. Subjects were categorized as NAFLD and non-NAFLD according to MR proton density fat fraction (PDFF) findings. Ultrasound images from 10 different locations in the right and left hepatic lobes were collected following a standard protocol. MRI-based liver fat quantification was used as the reference standard with >6.4% indicative of NAFLD. A supervised machine learning model was developed for assessment of NAFLD. To validate model performance, a balanced testing dataset of 24 subjects was used. Sensitivity, specificity, positive predictive value, negative predictive value, and overall accuracy with 95% confidence interval were calculated.

RESULTS

A total of 1119 images from 106 participants was used for model development. The internal evaluation achieved an average precision of 0.941, recall of 88.2%, and precision of 89.0%. In the testing set AutoML achieved a sensitivity of 72.2% (63.1%-80.1%), specificity of 94.6% (88.7%-98.0%), positive predictive value (PPV) of 93.1% (86.0%-96.7%), negative predictive value of 77.3% (71.6%-82.1%), and accuracy of 83.4% (77.9%-88.0%). The average agreement for an individual subject was 92%.

CONCLUSIONS

An ultrasound-based machine learning model for identification of NAFLD showed high specificity and PPV in this prospective trial. This approach may in the future be used as an inexpensive and noninvasive screening tool for identifying NAFLD in high-risk patients.

Two-dimensional Convolutional Neural Network Using Quantitative US for Noninvasive Assessment of Hepatic Steatosis in NAFLD

Background Quantitative US (QUS) using radiofrequency data analysis has been recently introduced for noninvasive evaluation of hepatic steatosis. Deep learning algorithms may improve the diagnostic performance of QUS for hepatic steatosis. Purpose To evaluate a two-dimensional (2D) convolutional neural network (CNN) algorithm using QUS parametric maps and B-mode images for diagnosis of hepatic steatosis, with the MRI-derived proton density fat fraction (PDFF) as the reference standard, in patients with nonalcoholic fatty liver disease (NAFLD). Materials and Methods: Consecutive adult participants with suspected NAFLD were prospectively enrolled at a single academic medical center from July 2020 to June 2021. Using radiofrequency data analysis, two QUS parameters (tissue attenuation imaging [TAI] and tissue scatter-distribution imaging [TSI]) were measured. On B-mode images, hepatic steatosis was graded using visual scoring (none, mild, moderate, or severe). Using B-mode images and two QUS parametric maps (TAI and TSI) as input data, the algorithm estimated the US fat fraction (USFF) as a percentage. The correlation between the USFF and MRI PDFF was evaluated using the Pearson correlation coefficient. The diagnostic performance of the USFF for hepatic steatosis (MRI PDFF ≥5%) was evaluated using receiver operating characteristic curve analysis and compared with that of TAI, TSI, and visual scoring. Results Overall, 173 participants (mean age, 51 years ± 14 [SD]; 96 men) were included, with 126 (73%) having hepatic steatosis (MRI PDFF ≥5%). USFF correlated strongly with MRI PDFF (Pearson r = 0.86, 95% CI: 0.82, 0.90; P < .001). For diagnosing hepatic steatosis (MRI PDFF ≥5%), the USFF yielded an area under the receiver operating characteristic curve of 0.97 (95% CI: 0.93, 0.99), higher than those of TAI, TSI, and visual scoring (P = .015, .006, and < .001, respectively), with a sensitivity of 90% (95% CI: 84, 95 [114 of 126]) and a specificity of 91% (95% CI: 80, 98 [43 of 47]) at a cutoff value of 5.7%. Conclusion A deep learning algorithm using quantitative US parametric maps and B-mode images accurately estimated the hepatic fat fraction and diagnosed hepatic steatosis in participants with nonalcoholic fatty liver disease. ClinicalTrials.gov registration nos. NCT04462562, NCT04180631 © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Sidhu and Fang in this issue.

Deep Learning for FAST Quality Assessment

OBJECTIVES

To determine the feasibility of using a deep learning (DL) algorithm to assess the quality of focused assessment with sonography in trauma (FAST) exams.

METHODS

Our dataset consists of 441 FAST exams, classified as good-quality or poor-quality, with 3161 videos. We first used convolutional neural networks (CNNs), pretrained on the Imagenet dataset and fine-tuned on the FAST dataset. Second, we trained a CNN autoencoder to compress FAST images, with a 20-1 compression ratio. The compressed codes were input to a two-layer classifier network. To train the networks, each video was labeled with the quality of the exam, and the frames were labeled with the quality of the video. For inference, a video was classified as poor-quality if half the frames were classified as poor-quality by the network, and an exam was classified as poor-quality if half the videos were classified as poor-quality.

RESULTS

The results with the encoder-classifier networks were much better than the transfer learning results with CNNs. This was primarily because the Imagenet dataset is not a good match for the ultrasound quality assessment problem. The DL models produced video sensitivities and specificities of 99% and 98% on held-out test sets.

CONCLUSIONS

Using an autoencoder to compress FAST images is a very effective way to obtain features that can be used to predict exam quality. These features are more suitable than those obtained from CNNs pretrained on Imagenet.