Transfer learning radiomics based on multimodal ultrasound imaging for staging liver fibrosis.Eur Radiol. 2020;30:2973-2983. [PMID: 31965257 DOI: 10.1007/s00330-019-06595-w]

Deep learning Radiomics Based on Two-Dimensional Ultrasound for Predicting the Efficacy of Neoadjuvant Chemotherapy in Breast Cancer

We investigate the predictive value of a comprehensive model based on preoperative ultrasound radiomics, deep learning, and clinical features for pathological complete response (pCR) after neoadjuvant chemotherapy (NAC) for the breast cancer. We enrolled 155 patients with pathologically confirmed breast cancer who underwent NAC. The patients were randomly divided into the training set and the validation set in the ratio of 7:3. The deep learning and radiomics features of pre-treatment ultrasound images were extracted, and the random forest recursive elimination algorithm and the least absolute shrinkage and selection operator were used for feature screening and DL-Score and Rad-Score construction. According to multifactorial logistic regression, independent clinical predictors, DL-Score, and Rad-Score were selected to construct the comprehensive prediction model DLRC. The performance of the model was evaluated in terms of its predictive effect, and clinical practicability. Compared to the clinical, radiomics (Rad-Score), and deep learning (DL-Score) models, the DLRC accurately predicted the pCR status, with an area under the curve (AUC) of 0.937 (95%CI: 0.895-0.970) in the training set and 0.914 (95%CI: 0.838-0.973) in the validation set. Moreover, decision curve analysis confirmed that the DLRC had the highest clinical value among all models. The comprehensive model DLRC based on ultrasound radiomics, deep learning, and clinical features can effectively and accurately predict the pCR status of breast cancer after NAC, which is conducive to assisting clinical personalized diagnosis and treatment plan.

Diagnostic value of ultrasound elastography combined with serological indicators in liver fibrosis in chronic hepatitis B

                                  To explore the value of ultrasound elastography combined with serological indicators in the diagnosis of liver fibrosis in chronic hepatitis B. A total of 156 patients with chronic hepatitis B from April 2020 to February 2022 were enrolled in this study as subjects. They were assigned to the liver fibrosis group (n=115) and the non-liver fibrosis group (n=41) according to whether the patients had liver fibrosis. They were divided into S1 stage (n=48), S2 stage (n=38), and S3 stage (n=29) according to histopathological staging criteria. Shear wave elastography (SWE) values, serum alanine aminotransferase (ALT), aspartate aminotransferase (AST), procollagen type III (PCIII), and laminin (LN) were compared among patients in each stage. Spearman's method was utilized to analyze the correlation of liver serum biochemical indicators and SWE value with liver fibrosis. The predictive performance of SWE value and serological indicators was analyzed using receiver operating characteristic curves. According to Spearman's method, the liver fibrosis stage was positively correlated with SWE value. Serological indicators combined with ultrasound elastography can accurately assess the degree of liver fibrosis in patients with chronic hepatitis B and provide a basis for clinical judgment.

Real-time carotid plaque recognition from dynamic ultrasound videos based on artificial neural network

PURPOSE

Carotid ultrasound allows noninvasive assessment of vascular anatomy and function with real-time display. Based on the transfer learning method, a series of research results have been obtained on the optimal image recognition and analysis of static images. However, for carotid plaque recognition, there are high requirements for self-developed algorithms in real-time ultrasound detection. This study aims to establish an automatic recognition system, Be Easy to Use (BETU), for the real-time and synchronous diagnosis of carotid plaque from ultrasound videos based on an artificial neural network.

MATERIALS AND METHODS

445 participants (mean age, 54.6±7.8 years; 227 men) were evaluated. Radiologists labeled a total of 3259 segmented ultrasound images from 445 videos with the diagnosis of carotid plaque, 2725 images were collected as a training dataset, and 554 images as a testing dataset. The automatic plaque recognition system BETU was established based on an artificial neural network, and remote application on a 5G environment was performed to test its diagnostic performance.

RESULTS

The diagnostic accuracy of BETU (98.5%) was consistent with the radiologist's (Kappa = 0.967, P < 0.001). Remote diagnostic feedback based on BETU-processed ultrasound videos could be obtained in 150ms across a distance of 1023 km between the ultrasound/BETU station and the consultation workstation.

CONCLUSION

Based on the good performance of BETU in real-time plaque recognition from ultrasound videos, 5G plus Artificial intelligence (AI)-assisted ultrasound real-time carotid plaque screening was achieved, and the diagnosis was made.

GLCV-NET: An automatic diagnosis system for advanced liver fibrosis using global-local cross view in B-mode ultrasound images

BACKGROUND AND OBJECTIVE

Advanced liver fibrosis is a critical stage in the evaluation of chronic liver disease (CLD), holding clinical significance in the development of treatment strategies and estimating the disease progression.

METHODS

This paper proposes an innovative Global-Local Cross-View Network (GLCV-Net) for the automatic diagnosis of advanced liver fibrosis from ultrasound (US) B-mode images. The proposed method consists of three main components: 1. A Segmentation-enhanced Global Hybrid Feature Extractor for segmenting the liver parenchyma and extracting global features; 2. A Heatmap-weighted Local Feature Extractor for selecting candidate regions and automatically identifying suspicious areas to construct local features; 3. A Scale-adaptive Fusion Module to balance the contributions of global and local scales in evaluating advanced liver fibrosis.

RESULTS

The predictive performance of the model was validated on an internal dataset of 1003 chronic liver disease (CLD) patients and an external dataset of 46 CLD patients, both subjected to liver fibrosis staging through pathological assessment. On the internal dataset, GLCV-Net achieved 86.9% accuracy, 85.0% recall, 85.4% precision, and 85.2% F1-score. Further validation on the external dataset confirmed its robustness, with scores of 86.1% in accuracy, 83.1% in recall, 80.8% in precision, and 81.9% in F1-score.

CONCLUSION

These results underscore the GLCV-Net's potential as a promising approach for non-invasively and accurately diagnosing advanced liver fibrosis in CLD patients, breaking through the limitations of traditional methods by integrating global and local information of liver fibrosis, significantly enhancing diagnostic accuracy.

Non-invasive diagnosis of liver fibrosis via MRI using targeted gadolinium-based nanoparticles

INTRODUCTION

Accurate diagnosis of liver fibrosis is crucial for preventing cirrhosis and liver tumors. Liver fibrosis is driven by activated hepatic stellate cells (HSCs) with elevated CD44 expression. We developed hyaluronic acid (HA)-coated gadolinium-based nanoprobes to specifically target CD44 for diagnosing liver fibrosis using T1-weighted magnetic resonance imaging (MRI).

MATERIALS AND METHODS

NaGdF4 nanoparticles (NPs) were synthesized via thermal decomposition and modified with polyethylene glycol (PEG) to obtain non-targeting NaGdF4@PEG NPs. These were subsequently coated with HA to target HSCs, resulting in liver fibrosis-targeting NaGdF4@PEG@HA nanoprobes. Characterization includedd transmission electron microscopy and X-ray diffraction. Cell viability was assessed using the Cell Counting Kit-8 (CCK-8). Internalization of NaGdF4@PEG@HA nanoprobes by mouse HSCs JS1 cells via ligand-receptor interaction was observed using flow cytometry and confocal laser scanning microscopy (CLSM). Liver fibrosis was induced in C57BL/6 mice using a methionine-choline deficient (MCD) diet. MRI performance and nanoprobe distribution in fibrotic and normal livers were analyzed using a GE Discovery 3.0T MR 750 scanner.

RESULTS

NaGdF4@PEG@HA nanoprobes exhibited homogeneous morphology, low toxicity, and a high T1 relaxation rate (7.645 mM⁻¹s⁻¹). CLSM and flow cytometry demonstrated effective phagocytosis of NaGdF4@PEG@HA nanoprobes by JS1 cells compared to NaGdF4@PEG. MRI scans revealed higher T1 signals in fibrotic livers compared to normal livers after injection of NaGdF4@PEG@HA. NaGdF4@PEG@HA demonstrated higher targeting ability in fibrotic mice.

CONCLUSIONS

NaGdF4@PEG@HA nanoprobes effectively target HSCs with high T1 relaxation rate, facilitating efficient MRI diagnosis of liver fibrosis.

Preoperative Ultrasound Radomics to Predict Posthepatectomy Liver Failure in Patients With Hepatocellular Carcinoma

PURPOSE

Posthepatectomy liver failure (PHLF) is a major cause of postoperative mortality in hepatocellular carcinoma (HCC) patients. The study aimed to develop a method based on the two-dimensional shear wave elastography and clinical data to evaluate the risk of PHLF in HCC patients with chronic hepatitis B.

METHODS

This multicenter study proposed a deep learning model (PHLF-Net) incorporating dual-modal ultrasound features and clinical indicators to predict the PHLF risk. The datasets were divided into a training cohort, an internal validation cohort, an internal independent testing cohort, and three external independent testing cohorts. Based on ResNet50 pretrained on ImageNet, PHLF-Net used a progressive training strategy with images of varying granularity and incorporated conventional B-mode and elastography images and clinical indicators related to liver reserve function.

RESULTS

In total, 532 HCC patients who underwent hepatectomy at five hospitals were enrolled. PHLF occurred in 147 patients (27.6%, 147/532). The PHLF-Net combining dual-modal ultrasound and clinical indicators demonstrated high effectiveness for predicting PHLF, with AUCs of 0.957 and 0.923 in the internal validation and testing sets, and AUCs of 0.950, 0.860, and 1.000 in the other three independent external testing sets. The performance of PHLF-Net outperformed models of single- and dual-modal US.

CONCLUSIONS

Preoperative ultrasound imaging combining clinical indicators can effectively predict the PHLF probability in patients with HCC. In the internal and external validation sets, PHLF-Net demonstrated its usefulness in predicting PHLF.

Machine learning model for non-alcoholic steatohepatitis diagnosis based on ultrasound radiomics

BACKGROUND

Non-Alcoholic Steatohepatitis (NASH) is a crucial stage in the progression of Non-Alcoholic Fatty Liver Disease(NAFLD). The purpose of this study is to explore the clinical value of ultrasound features and radiological analysis in predicting the diagnosis of Non-Alcoholic Steatohepatitis.

METHOD

An SD rat model of hepatic steatosis was established through a high-fat diet and subcutaneous injection of CCl4. Liver ultrasound images and elastography were acquired, along with serum data and histopathological results of rat livers.The Pyradiomics software was used to extract radiomic features from 2D ultrasound images of rat livers. The rats were then randomly divided into a training set and a validation set, and feature selection was performed through dimensionality reduction. Various machine learning (ML) algorithms were employed to build clinical diagnostic models, radiomic models, and combined diagnostic models. The efficiency of each diagnostic model for diagnosing NASH was evaluated using Receiver Operating Characteristic (ROC) curves, Clinical Decision Curve Analysis (DCA), and calibration curves.

RESULTS

In the machine learning radiomic model for predicting the diagnosis of NASH, the Area Under the Curve (AUC) of ROC curve for the clinical radiomic model in the training set and validation set were 0.989 and 0.885, respectively. The Decision Curve Analysis revealed that the clinical radiomic model had the highest net benefit within the probability threshold range of > 65%. The calibration curve in the validation set demonstrated that the clinical combined radiomic model is the optimal method for diagnosing Non-Alcoholic Steatohepatitis.

CONCLUSION

The combined diagnostic model constructed using machine learning algorithms based on ultrasound image radiomics has a high clinical predictive performance in diagnosing Non-Alcoholic Steatohepatitis.

Scaling-law mechanical marker for liver fibrosis diagnosis and drug screening through machine learning

Studies of cell and tissue mechanics have shown that significant changes in cell and tissue mechanics during lesions and cancers are observed, which provides new mechanical markers for disease diagnosis based on machine learning. However, due to the lack of effective mechanic markers, only elastic modulus and iconographic features are currently used as markers, which greatly limits the application of cell and tissue mechanics in disease diagnosis. Here, we develop a liver pathological state classifier through a support vector machine method, based on high dimensional viscoelastic mechanical data. Accurate diagnosis and grading of hepatic fibrosis facilitates early detection and treatment and may provide an assessment tool for drug development. To this end, we used the viscoelastic parameters obtained from the analysis of creep responses of liver tissues by a self-similar hierarchical model and built a liver state classifier based on machine learning. Using this classifier, we implemented a fast classification of healthy, diseased, and mesenchymal stem cells (MSCs)-treated fibrotic live tissues, and our results showed that the classification accuracy of healthy and diseased livers can reach 0.99, and the classification accuracy of the three liver tissues mixed also reached 0.82. Finally, we provide screening methods for markers in the context of massive data as well as high-dimensional viscoelastic variables based on feature ablation for drug development and accurate grading of liver fibrosis. We propose a novel classifier that uses the dynamical mechanical variables as input markers, which can identify healthy, diseased, and post-treatment liver tissues.

Development and Validation of an Ultrasonography-Based Machine Learning Model for Predicting Outcomes of Bruxism Treatments

BACKGROUND AND OBJECTIVES

We aimed to develop a predictive model for the outcome of bruxism treatments using ultrasonography (USG)-based machine learning (ML) techniques. This study is a quantitative research study (predictive modeling study) in which different treatment methods applied to bruxism patients are evaluated through artificial intelligence.

MATERIALS AND METHODS

The study population comprised 102 participants with bruxism in three treatment groups: Manual therapy, Manual therapy and Kinesio Tape or Botulinum Toxin-A injection. USG imaging was performed on the masseter muscle to calculate muscle thickness, and pain thresholds were evaluated using an algometer. A radiomics platform was utilized to handle imaging and clinical data, as well as to perform a subsequent radiomics statistical analysis.

RESULTS

The area under the curve (AUC) values of all machine learning methods ranged from 0.772 to 0.986 for the training data and from 0.394 to 0.848 for the test data. The Support Vector Machine (SVM) led to excellent discrimination between bruxism and normal patients from USG images. Radiomics characteristics in pre-treatment ultrasound scans of patients, showing coarse and nonuniform muscles, were associated with a greater chance of less effective pain reduction outcomes.

CONCLUSIONS

This study has introduced a machine learning model using SVM analysis on ultrasound (USG) images for bruxism patients, which can detect masseter muscle changes on USG. Support Vector Machine regression analysis showed the combined ML models can also predict the outcome of the pain reduction.

Role of radiomics in staging liver fibrosis: a meta-analysis

BACKGROUND

Fibrosis has important pathoetiological and prognostic roles in chronic liver disease. This study evaluates the role of radiomics in staging liver fibrosis.

METHOD

After literature search in electronic databases (Embase, Ovid, Science Direct, Springer, and Web of Science), studies were selected by following precise eligibility criteria. The quality of included studies was assessed, and meta-analyses were performed to achieve pooled estimates of area under receiver-operator curve (AUROC), accuracy, sensitivity, and specificity of radiomics in staging liver fibrosis compared to histopathology.

RESULTS

Fifteen studies (3718 patients; age 47 years [95% confidence interval (CI): 42, 53]; 69% [95% CI: 65, 73] males) were included. AUROC values of radiomics for detecting significant fibrosis (F2-4), advanced fibrosis (F3-4), and cirrhosis (F4) were 0.91 [95%CI: 0.89, 0.94], 0.92 [95%CI: 0.90, 0.95], and 0.94 [95%CI: 0.93, 0.96] in training cohorts and 0.89 [95%CI: 0.83, 0.91], 0.89 [95%CI: 0.83, 0.94], and 0.93 [95%CI: 0.91, 0.95] in validation cohorts, respectively. For diagnosing significant fibrosis, advanced fibrosis, and cirrhosis the sensitivity of radiomics was 84.0% [95%CI: 76.1, 91.9], 86.9% [95%CI: 76.8, 97.0], and 92.7% [95%CI: 89.7, 95.7] in training cohorts, and 75.6% [95%CI: 67.7, 83.5], 80.0% [95%CI: 70.7, 89.3], and 92.0% [95%CI: 87.8, 96.1] in validation cohorts, respectively. Respective specificity was 88.6% [95% CI: 83.0, 94.2], 88.4% [95% CI: 81.9, 94.8], and 91.1% [95% CI: 86.8, 95.5] in training cohorts, and 86.8% [95% CI: 83.3, 90.3], 94.0% [95% CI: 89.5, 98.4], and 88.3% [95% CI: 84.4, 92.2] in validation cohorts. Limitations included use of several methods for feature selection and classification, less availability of studies evaluating a particular radiological modality, lack of a direct comparison between radiology and radiomics, and lack of external validation.

CONCLUSION

Although radiomics offers good diagnostic accuracy in detecting liver fibrosis, its role in clinical practice is not as clear at present due to comparability and validation constraints.

Development and validation of an ultrasound-based deep learning radiomics nomogram for predicting the malignant risk of ovarian tumours

BACKGROUND

The timely identification and management of ovarian cancer are critical determinants of patient prognosis. In this study, we developed and validated a deep learning radiomics nomogram (DLR_Nomogram) based on ultrasound (US) imaging to accurately predict the malignant risk of ovarian tumours and compared the diagnostic performance of the DLR_Nomogram to that of the ovarian-adnexal reporting and data system (O-RADS).

METHODS

This study encompasses two research tasks. Patients were randomly divided into training and testing sets in an 8:2 ratio for both tasks. In task 1, we assessed the malignancy risk of 849 patients with ovarian tumours. In task 2, we evaluated the malignancy risk of 391 patients with O-RADS 4 and O-RADS 5 ovarian neoplasms. Three models were developed and validated to predict the risk of malignancy in ovarian tumours. The predicted outcomes of the models for each sample were merged to form a new feature set that was utilised as an input for the logistic regression (LR) model for constructing a combined model, visualised as the DLR_Nomogram. Then, the diagnostic performance of these models was evaluated by the receiver operating characteristic curve (ROC).

RESULTS

The DLR_Nomogram demonstrated superior predictive performance in predicting the malignant risk of ovarian tumours, as evidenced by area under the ROC curve (AUC) values of 0.985 and 0.928 for the training and testing sets of task 1, respectively. The AUC value of its testing set was lower than that of the O-RADS; however, the difference was not statistically significant. The DLR_Nomogram exhibited the highest AUC values of 0.955 and 0.869 in the training and testing sets of task 2, respectively. The DLR_Nomogram showed satisfactory fitting performance for both tasks in Hosmer-Lemeshow testing. Decision curve analysis demonstrated that the DLR_Nomogram yielded greater net clinical benefits for predicting malignant ovarian tumours within a specific range of threshold values.

CONCLUSIONS

The US-based DLR_Nomogram has shown the capability to accurately predict the malignant risk of ovarian tumours, exhibiting a predictive efficacy comparable to that of O-RADS.

Artificial intelligence in liver imaging: methods and applications

Liver disease is regarded as one of the major health threats to humans. Radiographic assessments hold promise in terms of addressing the current demands for precisely diagnosing and treating liver diseases, and artificial intelligence (AI), which excels at automatically making quantitative assessments of complex medical image characteristics, has made great strides regarding the qualitative interpretation of medical imaging by clinicians. Here, we review the current state of medical-imaging-based AI methodologies and their applications concerning the management of liver diseases. We summarize the representative AI methodologies in liver imaging with focusing on deep learning, and illustrate their promising clinical applications across the spectrum of precise liver disease detection, diagnosis and treatment. We also address the current challenges and future perspectives of AI in liver imaging, with an emphasis on feature interpretability, multimodal data integration and multicenter study. Taken together, it is revealed that AI methodologies, together with the large volume of available medical image data, might impact the future of liver disease care.

Automatic liver segmentation and assessment of liver fibrosis using deep learning with MR T1-weighted images in rats

OBJECTIVES

To validate the performance of nnU-Net in segmentation and CNN in classification for liver fibrosis using T1-weighted images.

MATERIALS AND METHODS

In this prospective study, animal models of liver fibrosis were induced by injecting subcutaneously a mixture of Carbon tetrachloride and olive oil. A total of 99 male Wistar rats were successfully induced and underwent MR scanning with no contrast agent to get T1-weighted images. The regions of interest (ROIs) of the whole liver were delineated layer by layer along the liver edge by 3D Slicer. For segmentation task, all T1-weighted images were randomly divided into training and test cohorts in a ratio of 7:3. For classification, images containing the hepatic maximum diameter of every rat were selected and 80% images of no liver fibrosis (NLF), early liver fibrosis (ELF) and progressive liver fibrosis (PLF) stages were randomly selected for training, while the rest were used for testing. Liver segmentation was performed by the nnU-Net model. The convolutional neural network (CNN) was used for classification task of liver fibrosis stages. The Dice similarity coefficient was used to evaluate the segmentation performance of nnU-Net. Confusion matrix, ROC curve and accuracy were used to show the classification performance of CNN.

RESULTS

A total of 2628 images were obtained from 99 Wistar rats by MR scanning. For liver segmentation by nnU-Net, the Dice similarity coefficient in the test set was 0.8477. The accuracies of CNN in staging NLF, ELF and PLF were 0.73, 0.89 and 0.84, respectively. The AUCs were 0.76, 0.88 and 0.79, respectively.

CONCLUSION

The nnU-Net architecture is of high accuracy for liver segmentation and CNN for assessment of liver fibrosis with T1-weighted images.

Ultrasound-Based Radiomics for the Classification of Henoch-Schönlein Purpura Nephritis in Children

Henoch-Schönlein purpura nephritis (HSPN) is one of the most common kidney diseases in children. The current diagnosis and classification of HSPN depend on pathological biopsy, which is seriously limited by its invasive and high-risk nature. The aim of the study was to explore the potential of radiomics model for evaluating the histopathological classification of HSPN based on the ultrasound (US) images. A total of 440 patients with Henoch-Schönlein purpura nephritis proved by biopsy were analyzed retrospectively. They were grouped according to two histopathological categories: those without glomerular crescent formation (ISKDC grades I-II) and those with glomerular crescent formation (ISKDC grades III-V). The patients were randomly assigned to either a training cohort (n = 308) or a validation cohort (n = 132) with a ratio of 7:3. The sonologist manually drew the regions of interest (ROI) on the ultrasound images of the right kidney including the cortex and medulla. Then, the ultrasound radiomics features were extracted using the Pyradiomics package. The dimensions of radiomics features were reduced by Spearman correlation coefficients and least absolute shrinkage and selection operator (LASSO) method. Finally, three radiomics models using k-nearest neighbor (KNN), logistic regression (LR), and support vector machine (SVM) were established, respectively. The predictive performance of such classifiers was assessed with receiver operating characteristic (ROC) curve. 105 radiomics features were extracted from derived US images of each patient and 14 features were ultimately selected for the machine learning analysis. Three machine learning models including k-nearest neighbor (KNN), logistic regression (LR), and support vector machine (SVM) were established for HSPN classification. Of the three classifiers, the SVM classifier performed the best in the validation cohort [area under the curve (AUC) =0.870 (95% CI, 0.795-0.944), sensitivity = 0.706, specificity = 0.950]. The US-based radiomics had good predictive value for HSPN classification, which can be served as a noninvasive tool to evaluate the severity of renal pathology and crescentic formation in children with HSPN.

Medical image identification methods: A review

The identification of medical images is an essential task in computer-aided diagnosis, medical image retrieval and mining. Medical image data mainly include electronic health record data and gene information data, etc. Although intelligent imaging provided a good scheme for medical image analysis over traditional methods that rely on the handcrafted features, it remains challenging due to the diversity of imaging modalities and clinical pathologies. Many medical image identification methods provide a good scheme for medical image analysis. The concepts pertinent of methods, such as the machine learning, deep learning, convolutional neural networks, transfer learning, and other image processing technologies for medical image are analyzed and summarized in this paper. We reviewed these recent studies to provide a comprehensive overview of applying these methods in various medical image analysis tasks, such as object detection, image classification, image registration, segmentation, and other tasks. Especially, we emphasized the latest progress and contributions of different methods in medical image analysis, which are summarized base on different application scenarios, including classification, segmentation, detection, and image registration. In addition, the applications of different methods are summarized in different application area, such as pulmonary, brain, digital pathology, brain, skin, lung, renal, breast, neuromyelitis, vertebrae, and musculoskeletal, etc. Critical discussion of open challenges and directions for future research are finally summarized. Especially, excellent algorithms in computer vision, natural language processing, and unmanned driving will be applied to medical image recognition in the future.

Application of artificial intelligence techniques for non-alcoholic fatty liver disease diagnosis: A systematic review (2005-2023)

BACKGROUND AND OBJECTIVES

Non-alcoholic fatty liver disease (NAFLD) is a common liver disease with a rapidly growing incidence worldwide. For prognostication and therapeutic decisions, it is important to distinguish the pathological stages of NAFLD: steatosis, steatohepatitis, and liver fibrosis, which are definitively diagnosed on invasive biopsy. Non-invasive ultrasound (US) imaging, including US elastography technique, and clinical parameters can be used to diagnose and grade NAFLD and its complications. Artificial intelligence (AI) is increasingly being harnessed for developing NAFLD diagnostic models based on clinical, biomarker, or imaging data. In this work, we systemically reviewed the literature for AI-enabled NAFLD diagnostic models based on US (including elastography) and clinical (including serological) data.

METHODS

We performed a comprehensive search on Google Scholar, Scopus, and PubMed search engines for articles published between January 2005 and June 2023 related to AI models for NAFLD diagnosis based on US and/or clinical parameters using the following search terms: "non-alcoholic fatty liver disease", "non-alcoholic steatohepatitis", "deep learning", "machine learning", "artificial intelligence", "ultrasound imaging", "sonography", "clinical information".

RESULTS

We reviewed 64 published models that used either US (including elastography) or clinical data input to detect the presence of NAFLD, non-alcoholic steatohepatitis, and/or fibrosis, and in some cases, the severity of steatosis, inflammation, and/or fibrosis as well. The performances of the published models were summarized, and stratified by data input and algorithms used, which could be broadly divided into machine and deep learning approaches.

CONCLUSION

AI models based on US imaging and clinical data can reliably detect NAFLD and its complications, thereby reducing diagnostic costs and the need for invasive liver biopsy. The models offer advantages of efficiency, accuracy, and accessibility, and serve as virtual assistants for specialists to accelerate disease diagnosis and reduce treatment costs for patients and healthcare systems.

CT-based intratumoral and peritumoral deep transfer learning features prediction of lymph node metastasis in non-small cell lung cancer

BACKGROUND

The main metastatic route for lung cancer is lymph node metastasis, and studies have shown that non-small cell lung cancer (NSCLC) has a high risk of lymph node infiltration.

OBJECTIVE

This study aimed to compare the performance of handcrafted radiomics (HR) features and deep transfer learning (DTL) features in Computed Tomography (CT) of intratumoral and peritumoral regions in predicting the metastatic status of NSCLC lymph nodes in different machine learning classifier models.

METHODS

We retrospectively collected data of 199 patients with pathologically confirmed NSCLC. All patients were divided into training (n = 159) and validation (n = 40) cohorts, respectively. The best HR and DTL features in the intratumoral and peritumoral regions were extracted and selected, respectively. Support Vector Machine (SVM), k-Nearest Neighbors (KNN), Light Gradient Boosting Machine (Light GBM), Multilayer Perceptron (MLP), and Logistic Regression (LR) models were constructed, and the performance of the models was evaluated.

RESULTS

Among the five models in the training and validation cohorts, the LR classifier model performed best in terms of HR and DTL features. The AUCs of the training cohort were 0.841 (95% CI: 0.776-0.907) and 0.955 (95% CI: 0.926-0.983), and the AUCs of the validation cohort were 0.812 (95% CI: 0.677-0.948) and 0.893 (95% CI: 0.795-0.991), respectively. The DTL signature was superior to the handcrafted radiomics signature.

CONCLUSIONS

Compared with the radiomics signature, the DTL signature constructed based on intratumoral and peritumoral areas in CT can better predict NSCLC lymph node metastasis.

Liver fibrosis classification from ultrasound using machine learning: a systematic literature review

PURPOSE

Liver biopsy was considered the gold standard for diagnosing liver fibrosis; however, with advancements in medical technology and increasing awareness of potential complications, the reliance on liver biopsy has diminished. Ultrasound is gaining popularity due to its wider availability and cost-effectiveness. This study examined the machine learning / deep learning (ML/DL) models for non-invasive liver fibrosis classification from ultrasound.

METHODS

Following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) protocol, we searched five academic databases using the query. We defined population, intervention, comparison, outcomes, and study design (PICOS) framework for the inclusion. Furthermore, Joana Briggs Institute (JBI) checklist for analytical cross-sectional studies is used for quality assessment.

RESULTS

Among the 188 screened studies, 17 studies are selected. The methods are categorized as off-the-shelf (OTS), attention, generative, and ensemble classifiers. Most studies used OTS classifiers that combined pre-trained ML/DL methods with radiomics features to determine fibrosis staging. Although machine learning shows potential for fibrosis classification, there are limited external comparisons of interventions and prospective clinical trials, which limits their applicability.

CONCLUSION

With the recent success of ML/DL toward biomedical image analysis, automated solutions using ultrasound are developed for predicting liver diseases. However, their applicability is bounded by the limited and imbalanced retrospective studies having high heterogeneity. This challenge could be addressed by generating a standard protocol for study design by selecting appropriate population, interventions, outcomes, and comparison.

Shear wave elastography-based deep learning model for prognosis of patients with acutely decompensated cirrhosis

PURPOSE

This study aimed to develop and validate a deep learning model based on two-dimensional (2D) shear wave elastography (SWE) for predicting prognosis in patients with acutely decompensated cirrhosis.

METHODS

We prospectively enrolled 288 acutely decompensated cirrhosis patients with a minimum 1-year follow-up, divided into a training cohort (202 patients, 1010 2D SWE images) and a test cohort (86 patients, 430 2D SWE images). Using transfer learning by Resnet-50 to analyze 2D SWE images, a SWE-based deep learning signature (DLswe) was developed for 1-year mortality prediction. A combined nomogram was established by incorporating deep learning SWE information and laboratory data through a multivariate Cox regression analysis. The performance of the nomogram was evaluated with respect to predictive discrimination, calibration, and clinical usefulness in the training and test cohorts.

RESULTS

The C-index for DLswe was 0.748 (95% CI 0.666-0.829) and 0.744 (95% CI 0.623-0.864) in the training and test cohorts, respectively. The combined nomogram significantly improved the C-index, accuracy, sensitivity, and specificity of DLswe to 0.823 (95% CI 0.763-0.883), 86%, 75%, and 89% in the training cohort, and 0.808 (95% CI 0.707-0.909), 83%, 74%, and 85% in the test cohort (both p < 0.05). Calibration curves demonstrated good calibration of the combined nomogram. Decision curve analysis indicated that the nomogram was clinically valuable.

CONCLUSIONS

The 2D SWE-based deep learning model holds promise as a noninvasive tool to capture valuable prognostic information, thereby improving outcome prediction in patients with acutely decompensated cirrhosis.

Application and prospects of AI-based radiomics in ultrasound diagnosis

Artificial intelligence (AI)-based radiomics has attracted considerable research attention in the field of medical imaging, including ultrasound diagnosis. Ultrasound imaging has unique advantages such as high temporal resolution, low cost, and no radiation exposure. This renders it a preferred imaging modality for several clinical scenarios. This review includes a detailed introduction to imaging modalities, including Brightness-mode ultrasound, color Doppler flow imaging, ultrasound elastography, contrast-enhanced ultrasound, and multi-modal fusion analysis. It provides an overview of the current status and prospects of AI-based radiomics in ultrasound diagnosis, highlighting the application of AI-based radiomics to static ultrasound images, dynamic ultrasound videos, and multi-modal ultrasound fusion analysis.

Artificial intelligence applications in computed tomography in gastric cancer: a narrative review

Background and Objective

Artificial intelligence (AI) is a revolutionary technique which is deeply impacting and reshaping clinical practice in oncology. This review aims to summarize the current status of the clinical application of AI-based computed tomography (CT) for gastric cancer (GC), focusing on diagnosis, genetic status detection and risk prediction of metastasis, prognosis and treatment efficacy. The challenges and prospects for future research will also be discussed.

Methods

We searched the PubMed/MEDLINE database to identify clinical studies published between 1990 and November 2022 that investigated AI applications in CT in GC. The major findings of the verified studies were summarized.

Key Content and Findings

AI applications in CT images have attracted considerable attention in various fields such as diagnosis, prediction of metastasis risk, survival, and treatment response. These emerging techniques have shown a high potential to outperform clinicians in diagnostic accuracy and time-saving.

Conclusions

AI-powered tools showed great potential to increase diagnostic accuracy and reduce radiologists' workload. However, the goal of AI is not to replace human ability but to help oncologists make decisions in their practice. Therefore, radiologists should play a predominant role in AI applications and decide the best ways to integrate these complementary techniques within clinical practice.

Radiological Diagnosis of Chronic Liver Disease and Hepatocellular Carcinoma: A Review

Medical image analysis plays a pivotal role in the evaluation of diseases, including screening, surveillance, diagnosis, and prognosis. Liver is one of the major organs responsible for key functions of metabolism, protein and hormone synthesis, detoxification, and waste excretion. Patients with advanced liver disease and Hepatocellular Carcinoma (HCC) are often asymptomatic in the early stages; however delays in diagnosis and treatment can lead to increased rates of decompensated liver diseases, late-stage HCC, morbidity and mortality. Ultrasound (US) is commonly used imaging modality for diagnosis of chronic liver diseases that includes fibrosis, cirrhosis and portal hypertension. In this paper, we first provide an overview of various diagnostic methods for stages of liver diseases and discuss the role of Computer-Aided Diagnosis (CAD) systems in diagnosing liver diseases. Second, we review the utility of machine learning and deep learning approaches as diagnostic tools. Finally, we present the limitations of existing studies and outline future directions to further improve diagnostic accuracy, as well as reduce cost and subjectivity, while also improving workflow for the clinicians.

A Hierarchical Siamese Network for Noninvasive Staging of Liver Fibrosis Based on US Image Pairs of the Liver and Spleen

Due to the heterogeneity of ultrasound (US) images and the indeterminate US texture of liver fibrosis (LF), automatic evaluation of LF based on US images is still challenging. Thus, this study aimed to propose a hierarchical Siamese network that combines the information from liver and spleen US images to improve the accuracy of LF grading. There were two stages in the proposed method. In stage one, a dual-channel Siamese network was trained to extract features from paired liver and spleen patches that were cropped from US images to avoid vascular interferences. Subsequently, the L1 distance was used to quantify the liver-spleen differences (LSDs). In stage two, the pretrained weights from stage one were transferred into the Siamese feature extractor of the LF staging model, and a classifier was trained using the fusion of the liver and LSD features for LF staging. This study was retrospectively conducted on US images of 286 patients with histologically proven liver fibrosis stages. Our method achieved a precision and sensitivity of 93.92% and 91.65%, respectively, for cirrhosis (S4) diagnosis, which is about 8% higher than that of the baseline model. The accuracy of the advanced fibrosis (≥S3) diagnosis and the multi-staging of fibrosis (≤S2 vs. S3 vs. S4) both improved about 5% to reach 90.40% and 83.93%, respectively. This study proposed a novel method that combined hepatic and splenic US images and improved the accuracy of LF staging, which indicates the great potential of liver-spleen texture comparison in noninvasive assessment of LF based on US images.

Artificial intelligence - based ultrasound elastography for disease evaluation - a narrative review

Ultrasound elastography (USE) provides complementary information of tissue stiffness and elasticity to conventional ultrasound imaging. It is noninvasive and free of radiation, and has become a valuable tool to improve diagnostic performance with conventional ultrasound imaging. However, the diagnostic accuracy will be reduced due to high operator-dependence and intra- and inter-observer variability in visual observations of radiologists. Artificial intelligence (AI) has great potential to perform automatic medical image analysis tasks to provide a more objective, accurate and intelligent diagnosis. More recently, the enhanced diagnostic performance of AI applied to USE have been demonstrated for various disease evaluations. This review provides an overview of the basic concepts of USE and AI techniques for clinical radiologists and then introduces the applications of AI in USE imaging that focus on the following anatomical sites: liver, breast, thyroid and other organs for lesion detection and segmentation, machine learning (ML) - assisted classification and prognosis prediction. In addition, the existing challenges and future trends of AI in USE are also discussed.

Diagnostic accuracy of ultrasound-based multimodal radiomics modeling for fibrosis detection in chronic kidney disease

OBJECTIVES

To predict kidney fibrosis in patients with chronic kidney disease using radiomics of two-dimensional ultrasound (B-mode) and Sound Touch Elastography (STE) images in combination with clinical features.

METHODS

The Mindray Resona 7 ultrasonic diagnostic apparatus with SC5-1U convex array probe (bandwidth frequency of 1-5 MHz) was used to perform two-dimensional ultrasound and STE software. The severity of cortical tubulointerstitial fibrosis was divided into three grades: mild interstitial fibrosis and tubular atrophy (IFTA), fibrotic area < 25%; moderate IFTA, fibrotic area 26-50%; and severe IFTA, fibrotic area > 50%. After extracting radiomics from B-mode and STE images in these patients, we analyzed two classification schemes: mild versus moderate-to-severe IFTA, and mild-to-moderate versus severe IFTA. A nomogram was constructed based on multiple logistic regression analyses, combining clinical and radiomics. The performance of the nomogram for differentiation was evaluated using receiver operating characteristic (ROC), calibration, and decision curves.

RESULTS

A total of 150 patients undergoing kidney biopsy were enrolled (mild IFTA: n = 74; moderate IFTA: n = 33; severe IFTA: n = 43) and randomized into training (n = 105) and validation cohorts (n = 45). To differentiate between mild and moderate-to-severe IFTA, a nomogram incorporating STE radiomics, albumin, and estimated glomerular filtration (eGFR) rate achieved an area under the ROC curve (AUC) of 0.91 (95% confidence interval [CI]: 0.85-0.97) and 0.85 (95% CI: 0.77-0.98) in the training and validation cohorts, respectively. Between mild-to-moderate and severe IFTA, the nomogram incorporating B-mode and STE radiomics features, age, and eGFR achieved an AUC of 0.93 (95% CI: 0.89-0.98) and 0.83 (95% CI: 0.70-0.95) in the training and validation cohorts, respectively. Finally, we performed a decision curve analysis and found that the nomogram using both radiomics and clinical features exhibited better predictability than any other model (DeLong test, p < 0.05 for the training and validation cohorts).

CONCLUSION

A nomogram based on two-dimensional ultrasound and STE radiomics and clinical features served as a non-invasive tool capable of differentiating kidney fibrosis of different severities.

KEY POINTS

• Radiomics calculated based on the ultrasound imaging may be used to predict the severities of kidney fibrosis. • Radiomics may be used to identify clinical features associated with the progression of tubulointerstitial fibrosis in patients with CKD. • Non-invasive ultrasound imaging-based radiomics method with accuracy aids in detecting renal fibrosis with different IFTA severities.

A comprehensive survey of deep learning research on medical image analysis with focus on transfer learning

This survey aims to identify commonly used methods, datasets, future trends, knowledge gaps, constraints, and limitations in the field to provide an overview of current solutions used in medical image analysis in parallel with the rapid developments in transfer learning (TL). Unlike previous studies, this survey grouped the last five years of current studies for the period between January 2017 and February 2021 according to different anatomical regions and detailed the modality, medical task, TL method, source data, target data, and public or private datasets used in medical imaging. Also, it provides readers with detailed information on technical challenges, opportunities, and future research trends. In this way, an overview of recent developments is provided to help researchers to select the most effective and efficient methods and access widely used and publicly available medical datasets, research gaps, and limitations of the available literature.

Capsule robot pose and mechanism state detection in ultrasound using attention-based hierarchical deep learning

Ingestible robotic capsules with locomotion capabilities and on-board sampling mechanism have great potential for non-invasive diagnostic and interventional use in the gastrointestinal tract. Real-time tracking of capsule location and operational state is necessary for clinical application, yet remains a significant challenge. To this end, we propose an approach that can simultaneously determine the mechanism state and in-plane 2D pose of millimeter capsule robots in an anatomically representative environment using ultrasound imaging. Our work proposes an attention-based hierarchical deep learning approach and adapts the success of transfer learning towards solving the multi-task tracking problem with limited dataset. To train the neural networks, we generate a representative dataset of a robotic capsule within ex-vivo porcine stomachs. Experimental results show that the accuracy of capsule state classification is 97%, and the mean estimation errors for orientation and centroid position are 2.0 degrees and 0.24 mm (1.7% of the capsule's body length) on the hold-out test set. Accurate detection of the capsule while manipulated by an external magnet in a porcine stomach and colon is also demonstrated. The results suggest our proposed method has the potential for advancing the wireless capsule-based technologies by providing accurate detection of capsule robots in clinical scenarios.

Ultrasound-based artificial intelligence in gastroenterology and hepatology

Artificial intelligence (AI), especially deep learning, is gaining extensive attention for its excellent performance in medical image analysis. It can automatically make a quantitative assessment of complex medical images and help doctors to make more accurate diagnoses. In recent years, AI based on ultrasound has been shown to be very helpful in diffuse liver diseases and focal liver lesions, such as analyzing the severity of nonalcoholic fatty liver and the stage of liver fibrosis, identifying benign and malignant liver lesions, predicting the microvascular invasion of hepatocellular carcinoma, curative transarterial chemoembolization effect, and prognoses after thermal ablation. Moreover, AI based on endoscopic ultrasonography has been applied in some gastrointestinal diseases, such as distinguishing gastric mesenchymal tumors, detection of pancreatic cancer and intraductal papillary mucinous neoplasms, and predicting the preoperative tumor deposits in rectal cancer. This review focused on the basic technical knowledge about AI and the clinical application of AI in ultrasound of liver and gastroenterology diseases. Lastly, we discuss the challenges and future perspectives of AI.

Artificial Intelligence-Based Feature Analysis of Ultrasound Images of Liver Fibrosis

Liver fibrosis is a common liver disease that seriously endangers human health. Liver biopsy is the gold standard for diagnosing liver fibrosis, but its clinical use is limited due to its invasive nature. Ultrasound image examination is a widely used liver fibrosis examination method. Clinicians can diagnose the severity of liver fibrosis according to their own experience by observing the roughness of the texture of the ultrasound image, and this method is highly subjective. Under the premise that artificial intelligence technology is widely used in medical image analysis, this paper uses convolutional neural network analysis to extract the characteristics of ultrasound images of liver fibrosis and then classify the degree of liver fibrosis. Using neural network for image classification can avoid the subjectivity of manual classification and improve the accuracy of judging the degree of liver fibrosis, so as to complete the prevention and treatment of liver fibrosis. Therefore, the following work is done in this paper: (1) the research background, research significance, research status at home and abroad, and the impact of the development of medical imaging on the diagnosis of liver fibrosis are introduced; (2) the related technologies of deep learning and deep convolutional network are introduced, and the indicators of liver fibrosis degree assessment are constructed by using ultrasonic image extraction features; (3) using the collected liver fibrosis dataset to conduct model evaluation experiments, four classic CNN models are selected to compare and analyze the recognition rate. The experiments show that the GoogLeNet model has the best classification and recognition effect.

Deep learning in ultrasound elastography imaging: A review

It is known that changes in the mechanical properties of tissues are associated with the onset and progression of certain diseases. Ultrasound elastography is a technique to characterize tissue stiffness using ultrasound imaging either by measuring tissue strain using quasi-static elastography or natural organ pulsation elastography, or by tracing a propagated shear wave induced by a source or a natural vibration using dynamic elastography. In recent years, deep learning has begun to emerge in ultrasound elastography research. In this review, several common deep learning frameworks in the computer vision community, such as multilayer perceptron, convolutional neural network, and recurrent neural network are described. Then, recent advances in ultrasound elastography using such deep learning techniques are revisited in terms of algorithm development and clinical diagnosis. Finally, the current challenges and future developments of deep learning in ultrasound elastography are prospected. This article is protected by copyright. All rights reserved.

Image Human Thorax Using Ultrasound Traveltime Tomography with Supervised Descent Method

The change of acoustic velocity in the human thorax reflects the functional status of the respiratory system. Imaging the thorax’s acoustic velocity distribution can be used to monitor the respiratory system. In this paper, the feasibility of imaging the human thorax using ultrasound traveltime tomography with a supervised descent method (SDM) is studied. The forward modeling is computed using the shortest path ray tracing (SPR) method. The training model is composed of homogeneous acoustic velocity background and a high-velocity rectangular block moving in the domain of interest (DoI). The average descent direction is learned from the training set. Numerical experiments are conducted to verify the method’s feasibility. Normal thorax model experiment proves that SDM traveltime tomography can efficiently reconstruct thorax acoustic velocity distribution. Numerical experiments based on synthetic thorax model of pleural effusion and pneumothorax show that SDM traveltime tomography has good generalization ability and can detect the change of acoustic velocity in human thorax. This method might be helpful for the diagnosis and evaluation of respiratory diseases.

Ultrasound-Based Radiomics Can Classify the Etiology of Cervical Lymphadenopathy: A Multi-Center Retrospective Study

Medical diagnostic imaging is essential for the differential diagnosis of cervical lymphadenopathy. Here we develop an ultrasound radiomics method for accurately differentiating cervical lymph node tuberculosis (LNTB), cervical lymphoma, reactive lymph node hyperplasia, and metastatic lymph nodes especially in the multi-operator, cross-machine, multicenter context. The inter-observer and intra-observer consistency of radiomics parameters from the region of interest were 0.8245 and 0.9228, respectively. The radiomics model showed good and repeatable diagnostic performance for multiple classification diagnosis of cervical lymphadenopathy, especially in LNTB (area under the curve, AUC: 0.673, 0.662, and 0.626) and cervical lymphoma (AUC: 0.623, 0.644, and 0.602) in the whole set, training set, and test set, respectively. However, the diagnostic performance of lymphadenopathy among skilled radiologists was varied (Kappa coefficient: 0.108, *p < 0.001). The diagnostic performance of radiomics is comparable and more reproducible compared with those of skilled radiologists. Our study offers a more comprehensive method for differentiating LNTB, cervical lymphoma, reactive lymph node hyperplasia, and metastatic LN.

Conventional and artificial intelligence-based imaging for biomarker discovery in chronic liver disease

Chronic liver diseases, resulting from chronic injuries of various causes, lead to cirrhosis with life-threatening complications including liver failure, portal hypertension, hepatocellular carcinoma. A key unmet medical need is robust non-invasive biomarkers to predict patient outcome, stratify patients for risk of disease progression and monitor response to emerging therapies. Quantitative imaging biomarkers have already been developed, for instance, liver elastography for staging fibrosis or proton density fat fraction on magnetic resonance imaging for liver steatosis. Yet, major improvements, in the field of image acquisition and analysis, are still required to be able to accurately characterize the liver parenchyma, monitor its changes and predict any pejorative evolution across disease progression. Artificial intelligence has the potential to augment the exploitation of massive multi-parametric data to extract valuable information and achieve precision medicine. Machine learning algorithms have been developed to assess non-invasively certain histological characteristics of chronic liver diseases, including fibrosis and steatosis. Although still at an early stage of development, artificial intelligence-based imaging biomarkers provide novel opportunities to predict the risk of progression from early-stage chronic liver diseases toward cirrhosis-related complications, with the ultimate perspective of precision medicine. This review provides an overview of emerging quantitative imaging techniques and the application of artificial intelligence for biomarker discovery in chronic liver disease.

Transfer learning for medical image classification: a literature review

BACKGROUND

Transfer learning (TL) with convolutional neural networks aims to improve performances on a new task by leveraging the knowledge of similar tasks learned in advance. It has made a major contribution to medical image analysis as it overcomes the data scarcity problem as well as it saves time and hardware resources. However, transfer learning has been arbitrarily configured in the majority of studies. This review paper attempts to provide guidance for selecting a model and TL approaches for the medical image classification task.

METHODS

425 peer-reviewed articles were retrieved from two databases, PubMed and Web of Science, published in English, up until December 31, 2020. Articles were assessed by two independent reviewers, with the aid of a third reviewer in the case of discrepancies. We followed the PRISMA guidelines for the paper selection and 121 studies were regarded as eligible for the scope of this review. We investigated articles focused on selecting backbone models and TL approaches including feature extractor, feature extractor hybrid, fine-tuning and fine-tuning from scratch.

RESULTS

The majority of studies (n = 57) empirically evaluated multiple models followed by deep models (n = 33) and shallow (n = 24) models. Inception, one of the deep models, was the most employed in literature (n = 26). With respect to the TL, the majority of studies (n = 46) empirically benchmarked multiple approaches to identify the optimal configuration. The rest of the studies applied only a single approach for which feature extractor (n = 38) and fine-tuning from scratch (n = 27) were the two most favored approaches. Only a few studies applied feature extractor hybrid (n = 7) and fine-tuning (n = 3) with pretrained models.

CONCLUSION

The investigated studies demonstrated the efficacy of transfer learning despite the data scarcity. We encourage data scientists and practitioners to use deep models (e.g. ResNet or Inception) as feature extractors, which can save computational costs and time without degrading the predictive power.

Prediction of programmed cell death protein 1 in hepatocellular carcinoma patients using radiomics analysis with radiofrequency-based ultrasound multifeature maps

Background

This study explored the feasibility of radiofrequency (RF)-based radiomics analysis techniques for the preoperative prediction of programmed cell death protein 1 (PD-1) in patients with hepatocellular carcinoma (HCC).

Methods

The RF-based radiomics analysis method used ultrasound multifeature maps calculated from the RF signals of HCC patients, including direct energy attenuation (DEA) feature map, skewness of spectrum difference (SSD) feature map, and noncentrality parameter S of the Rician distribution (NRD) feature map. From each of the above ultrasound maps, 345 high-throughput radiomics features were extracted. Then, the useful radiomics features were selected by the sparse representation method and input into support vector machine (SVM) classifier for PD-1 prediction.

Results and conclusion

Among all the RF-based prediction models and the ultrasound grayscale comparative model, the RF-based model using all of the three ultrasound feature maps had the highest prediction accuracy (ACC) and area under the curve (AUC), which were 92.5% and 94.23%, respectively. The method proposed in this paper is effective for the meaningful feature extraction of RF signals and can effectively predict PD-1 in patients with HCC.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12938-021-00927-y.

We proposed RF-based radiomics analysis method by introducing three ultrasound features of direct energy attenuation (DEA), skewness of spectrum difference (SSD) and noncentrality parameter S of Rician distribution (NRD) as the feature extraction method from RF signals, investigated the effectiveness of RF-based radiomics analysis method in the immunocheckpoint prediction of programmed cell death protein 1 (PD-1), and validated the results with contrast testing of grayscale-based radiomics analysis method in this study. We also demonstrate a trend in prediction performance changes and its correlation with the number of ultrasound features.

The results demonstrated that there were significant differences (p < 0.05) in radiomics scores between HCC patients with PD-1 and HCC patients without PD-1. RF-based radiomics analysis method performed well in PD-1 noninvasive preoperative prediction of HCC patients.

In this study, the performance of RF-based radiomics analysis method was better than that of grayscale-based radiomics analysis method in the preoperative prediction of PD-1 in HCC patients. The AUC of DSNM, which was the RF-based radiomics analysis model with three ultrasound feature maps, reached 94.23% in the prediction of PD-1 cell protein in HCC patients.

Development and validation of a composite AI model for the diagnosis of levator ani muscle avulsion

OBJECTIVE

To assess the feasibility and reliability of a composite AI model for the diagnosis of levator ani muscle (LAM) avulsion of tomographic ultrasound imaging (TUI).

METHODS

Ultrasonic images of the pelvic floor from a total of 304 patients taken from January 2018 to October 2020 were included. All patients included underwent standardized interviews and transperineal ultrasound (TPUS). Transfer-learning and ensemble-learning methods were adopted to develop the proposed model on the basis of three classic convolutional neural networks (CNN). Confusion matrix (CM) and the ROC statistic were used to assess the effectiveness of the proposed model. Gradient-weighted class activation mappings (Grad-CAMs) were used to help enhance the interpretability of the proposed model.

RESULTS

Of the 304 patients included, 208 were in the derivation cohort (108 LAM avulsion and 100 normal) and 96 (39 LAM avulsion and 57 normal) were in the validation cohort. The proposed model in LAM avulsion diagnosis outperformed other models and a junior clinician in both the test set of derivation cohort and the validation cohort, with accuracies of 0.95 and 0.81, and AUCs of 0.98 and 0.86, respectively. According to the heatmap of Grad-CAMs, the proposed model mainly localizes areas between the pubic symphysis and the bilateral insertion point of LAM when making a diagnosis, which is exactly the region of interest in clinical practice.

CONCLUSION

The proposed model using ultrasonic images of the pelvic floor may be a promising tool in assisting the diagnosis of LAM avulsion in clinical practice.

KEY POINTS

• First AI-assisted model for levator ani muscle avulsion diagnosis • Diagnosis accuracy of less-experienced clinicians could be improved using the proposed model.

Multi-scale information with attention integration for classification of liver fibrosis in B-mode US image

BACKGROUND AND OBJECTIVE

Chronic hepatitis B (CHB) is one of the most common liver diseases in the world, which threats a lot to people's usual life. The increased deposition of fibrotic tissues in livers for patients with CHB may lead to the development of liver cirrhosis, hepatocellular carcinoma, or even liver failure. Accurate fibrosis staging is very important for the targeted treatment of liver fibrosis and its recovery.

METHODS

In this paper, we propose a new deep convolutional neural network (DCNN) with functions of multi-scale information extraction and attention integration for more accurate liver fibrosis classification from ultrasound (US) images. The proposed network uses two pyramid-structured CNN elements to extract multi-scale features from US images. Such a design significantly enlarges the receptive field of the convolution layer, such that more useful information can be explored by the neural network to associate with the final classification. Based on this, a new feature distillation method is also proposed to enhance the ability of deep features derived from multi-scale information. The proposed distillation method employs attention maps to automatically extract class-related features from multi-scale information, which effectively suppress the influence of potential distractors.

RESULTS

Experimental results on the US liver fibrosis dataset collected from 286 participants show that the proposed deep framework achieves promising classification performance. The proposed method achieves a classification accuracy of 95.66% on the test dataset.

CONCLUSION

Our proposed framework could stage liver fibrosis highly accurately. It might provide effective suggestions for the clinical treatment of liver fibrosis that can facilitate its recovery.

The application of artificial intelligence in hepatology: A systematic review

The integration of human and artificial intelligence (AI) in medicine has only recently begun but it has already become obvious that intelligent systems can dramatically improve the management of liver diseases. Big data made it possible to envisage transformative developments of the use of AI for diagnosing, predicting prognosis and treating liver diseases, but there is still a lot of work to do. If we want to achieve the 21^st century digital revolution, there is an urgent need for specific national and international rules, and to adhere to bioethical parameters when collecting data. Avoiding misleading results is essential for the effective use of AI. A crucial question is whether it is possible to sustain, technically and morally, the process of integration between man and machine. We present a systematic review on the applications of AI to hepatology, highlighting the current challenges and crucial issues related to the use of such technologies.

Deep Learning With Radiomics for Disease Diagnosis and Treatment: Challenges and Potential

The high-throughput extraction of quantitative imaging features from medical images for the purpose of radiomic analysis, i.e., radiomics in a broad sense, is a rapidly developing and emerging research field that has been attracting increasing interest, particularly in multimodality and multi-omics studies. In this context, the quantitative analysis of multidimensional data plays an essential role in assessing the spatio-temporal characteristics of different tissues and organs and their microenvironment. Herein, recent developments in this method, including manually defined features, data acquisition and preprocessing, lesion segmentation, feature extraction, feature selection and dimension reduction, statistical analysis, and model construction, are reviewed. In addition, deep learning-based techniques for automatic segmentation and radiomic analysis are being analyzed to address limitations such as rigorous workflow, manual/semi-automatic lesion annotation, and inadequate feature criteria, and multicenter validation. Furthermore, a summary of the current state-of-the-art applications of this technology in disease diagnosis, treatment response, and prognosis prediction from the perspective of radiology images, multimodality images, histopathology images, and three-dimensional dose distribution data, particularly in oncology, is presented. The potential and value of radiomics in diagnostic and therapeutic strategies are also further analyzed, and for the first time, the advances and challenges associated with dosiomics in radiotherapy are summarized, highlighting the latest progress in radiomics. Finally, a robust framework for radiomic analysis is presented and challenges and recommendations for future development are discussed, including but not limited to the factors that affect model stability (medical big data and multitype data and expert knowledge in medical), limitations of data-driven processes (reproducibility and interpretability of studies, different treatment alternatives for various institutions, and prospective researches and clinical trials), and thoughts on future directions (the capability to achieve clinical applications and open platform for radiomics analysis).

Liver fibrosis staging by computed tomography: Prospective randomized multicentric evaluation of image analyses

AIM

Liver fibrosis staging is essential. We prospectively evaluated the liver fibrosis staging performance of computed tomography (CT).

METHODS

70 hepato-gastroenterology clinicians were randomized into three stratified groups with different image analyses of radiological semiology, i.e., on raw images (group 1) and on expert-annotated (group 2) and computerized-morphometry-enriched (group 3) images. Radiological fibrosis staging based on seven simple descriptors into four stages equivalent to Metavir stages (F0/1, F2, F3, F4=cirrhosis) was determined at baseline and after image analyses in 10 patients with chronic liver diseases (two per F) concordant for four independent fibrosis stagings including Metavir. 23,800 CT images were analysed, providing 1400 fibrosis stagings.

RESULTS

Fibrosis staging: overall (3 groups) accuracy (correct classification rate) was, baseline: 43%, post-analysis: 60% (p < 0.001) without significant progression in group 1 (6%, p = 0.207) contrary to groups 2 (34%, p < 0.001) and 3 (13%, p = 0.007). Cirrhosis diagnosis: overall accuracy was, baseline: 84%, post-analysis: 89% (p < 0.001) without significant progression in group 1 (0%, p = 1) contrary to groups 2 (8%, p = 0.009) and 3 (7%, p = 0.015). Baseline AUROCs were good (≥0.83) for marked fibrosis (F≥3 or cirrhosis) in all groups. Post-analysis AUROCs became excellent (≥0.89) in group 2 for all diagnostic targets (≥0.98 for F≥3 and cirrhosis) and in group 3 for cirrhosis. In post-analysis group 2, discrimination between all F was excellent (especially, F1 from F0) with an Obuchowski index at 0.87. Negative and positive predictive values for marked fibrosis were 98% and 95%, respectively.

CONCLUSION

Simple CT descriptors accurately discriminate all Metavir liver fibrosis stages.

A Transfer Learning Radiomics Nomogram for Preoperative Prediction of Borrmann Type IV Gastric Cancer From Primary Gastric Lymphoma

Objective

This study aims to differentiate preoperative Borrmann type IV gastric cancer (GC) from primary gastric lymphoma (PGL) by transfer learning radiomics nomogram (TLRN) with whole slide images of GC as source domain data.

Materials and Methods

This study retrospectively enrolled 438 patients with histopathologic diagnoses of Borrmann type IV GC and PGL. They received CT examinations from three hospitals. Quantitative transfer learning features were extracted by the proposed transfer learning radiopathomic network and used to construct transfer learning radiomics signatures (TLRS). A TLRN, which integrates TLRS, clinical factors, and CT subjective findings, was developed by multivariate logistic regression. The diagnostic TLRN performance was assessed by clinical usefulness in the independent validation set.

Results

The TLRN was built by TLRS and a high enhanced serosa sign, which showed good agreement by the calibration curve. The TLRN performance was superior to the clinical model and TLRS. Its areas under the curve (AUC) were 0.958 (95% confidence interval [CI], 0.883–0.991), 0.867 (95% CI, 0.794–0.922), and 0.921 (95% CI, 0.860–0.960) in the internal and two external validation cohorts, respectively. Decision curve analysis (DCA) showed that the TLRN was better than any other model. TLRN has potential generalization ability, as shown in the stratification analysis.

Conclusions

The proposed TLRN based on gastric WSIs may help preoperatively differentiate PGL from Borrmann type IV GC.

Borrmann type IV gastric cancer, primary gastric lymphoma, transfer learning, whole slide image, deep learning.

Differentiation of Brain Abscess From Cystic Glioma Using Conventional MRI Based on Deep Transfer Learning Features and Hand-Crafted Radiomics Features

Objectives: To develop and validate the model for distinguishing brain abscess from cystic glioma by combining deep transfer learning (DTL) features and hand-crafted radiomics (HCR) features in conventional T1-weighted imaging (T1WI) and T2-weighted imaging (T2WI).

Methods: This single-center retrospective analysis involved 188 patients with pathologically proven brain abscess (102) or cystic glioma (86). One thousand DTL and 105 HCR features were extracted from the T1WI and T2WI of the patients. Three feature selection methods and four classifiers, such as k-nearest neighbors (KNN), random forest classifier (RFC), logistic regression (LR), and support vector machine (SVM), for distinguishing brain abscess from cystic glioma were compared. The best feature combination and classifier were chosen according to the quantitative metrics including area under the curve (AUC), Youden Index, and accuracy.

Results: In most cases, deep learning-based radiomics (DLR) features, i.e., DTL features combined with HCR features, contributed to a higher accuracy than HCR and DTL features alone for distinguishing brain abscesses from cystic gliomas. The AUC values of the model established, based on the DLR features in T2WI, were 0.86 (95% CI: 0.81, 0.91) in the training cohort and 0.85 (95% CI: 0.75, 0.95) in the test cohort, respectively.

Conclusions: The model established with the DLR features can distinguish brain abscess from cystic glioma efficiently, providing a useful, inexpensive, convenient, and non-invasive method for differential diagnosis. This is the first time that conventional MRI radiomics is applied to identify these diseases. Also, the combination of HCR and DTL features can lead to get impressive performance.

Clinical Artificial Intelligence Applications in Radiology: Chest and Abdomen

Organ segmentation, chest radiograph classification, and lung and liver nodule detections are some of the popular artificial intelligence (AI) tasks in chest and abdominal radiology due to the wide availability of public datasets. AI algorithms have achieved performance comparable to humans in less time for several organ segmentation tasks, and some lesion detection and classification tasks. This article introduces the current published articles of AI applied to chest and abdominal radiology, including organ segmentation, lesion detection, classification, and predicting prognosis.

Deep learning for noninvasive liver fibrosis classification: A systematic review

BACKGROUND AND AIMS

While biopsy is the gold standard for liver fibrosis staging, it poses significant risks. Noninvasive assessment of liver fibrosis is a growing field. Recently, deep learning (DL) technology has revolutionized medical image analysis. This technology has the potential to enhance noninvasive fibrosis assessment. We systematically examined the application of DL in noninvasive liver fibrosis imaging.

METHODS

Embase, MEDLINE, Web of Science, and IEEE Xplore databases were used to identify studies that reported on the accuracy of DL for classification of liver fibrosis on noninvasive imaging. The search keywords were "liver or hepatic," "fibrosis or cirrhosis," and "neural or DL networks." Risk of bias and applicability were evaluated using the QUADAS-2 tool.

RESULTS

Sixteen studies were retrieved. Imaging modalities included ultrasound (n = 10), computed tomography (n = 3), and magnetic resonance imaging (n = 3). The studies analyzed a total of 40 405 radiological images from 15 853 patients. All but two of the studies were retrospective. In most studies the "ground truth" reference was the METAVIR score for pathological staging (n = 9.56%). The majority of the studies reported an accuracy >85% when compared to histopathology. Fourteen studies (87.5%) had a high risk of bias and concerns regarding applicability.

CONCLUSIONS

Deep learning has the potential to play an emerging role in liver fibrosis classification. Yet, it is still limited by a relatively small number of retrospective studies. Clinicians should facilitate the use of this technology by sharing databases and standardized reports. This may optimize the noninvasive evaluation of liver fibrosis on a large scale.

Artificial intelligence for hepatitis evaluation

Recently, increasing attention has been paid to the application of artificial intelligence (AI) to the diagnosis of diverse hepatic diseases, which comprises traditional machine learning and deep learning. Recent studies have shown the possible value of AI based data mining in predicting the incidence of hepatitis, classifying the different stages of hepatitis, diagnosing or screening for hepatitis, forecasting the progression of hepatitis, and predicting response to antiviral drugs in chronic hepatitis C patients. More importantly, AI based on radiology has been proven to be useful in predicting hepatitis and liver fibrosis as well as grading hepatocellular carcinoma (HCC) and differentiating it from benign liver tumors. It can predict the risk of vascular invasion of HCC, the risk of hepatic encephalopathy secondary to hepatitis B related cirrhosis, and the risk of liver failure after hepatectomy in HCC patients. In this review, we summarize the application of AI in hepatitis, and identify the challenges and future perspectives.

Future artificial intelligence tools and perspectives in medicine

PURPOSE OF REVIEW

Artificial intelligence has become popular in medical applications, specifically as a clinical support tool for computer-aided diagnosis. These tools are typically employed on medical data (i.e., image, molecular data, clinical variables, etc.) and used the statistical and machine-learning methods to measure the model performance. In this review, we summarized and discussed the most recent radiomic pipeline used for clinical analysis.

RECENT FINDINGS

Currently, limited management of cancers benefits from artificial intelligence, mostly related to a computer-aided diagnosis that avoids a biopsy analysis that presents additional risks and costs. Most artificial intelligence tools are based on imaging features, known as radiomic analysis that can be refined into predictive models in noninvasively acquired imaging data. This review explores the progress of artificial intelligence-based radiomic tools for clinical applications with a brief description of necessary technical steps. Explaining new radiomic approaches based on deep-learning techniques will explain how the new radiomic models (deep radiomic analysis) can benefit from deep convolutional neural networks and be applied on limited data sets.

SUMMARY

To consider the radiomic algorithms, further investigations are recommended to involve deep learning in radiomic models with additional validation steps on various cancer types.

Advances in Deep Learning-Based Medical Image Analysis

Importance. With the booming growth of artificial intelligence (AI), especially the recent advancements of deep learning, utilizing advanced deep learning-based methods for medical image analysis has become an active research area both in medical industry and academia. This paper reviewed the recent progress of deep learning research in medical image analysis and clinical applications. It also discussed the existing problems in the field and provided possible solutions and future directions.Highlights. This paper reviewed the advancement of convolutional neural network-based techniques in clinical applications. More specifically, state-of-the-art clinical applications include four major human body systems: the nervous system, the cardiovascular system, the digestive system, and the skeletal system. Overall, according to the best available evidence, deep learning models performed well in medical image analysis, but what cannot be ignored are the algorithms derived from small-scale medical datasets impeding the clinical applicability. Future direction could include federated learning, benchmark dataset collection, and utilizing domain subject knowledge as priors.Conclusion. Recent advanced deep learning technologies have achieved great success in medical image analysis with high accuracy, efficiency, stability, and scalability. Technological advancements that can alleviate the high demands on high-quality large-scale datasets could be one of the future developments in this area.

Detection of liver cirrhosis in standard T2-weighted MRI using deep transfer learning

Objectives

To investigate the diagnostic performance of deep transfer learning (DTL) to detect liver cirrhosis from clinical MRI.

Methods

The dataset for this retrospective analysis consisted of 713 (343 female) patients who underwent liver MRI between 2017 and 2019. In total, 553 of these subjects had a confirmed diagnosis of liver cirrhosis, while the remainder had no history of liver disease. T2-weighted MRI slices at the level of the caudate lobe were manually exported for DTL analysis. Data were randomly split into training, validation, and test sets (70%/15%/15%). A ResNet50 convolutional neural network (CNN) pre-trained on the ImageNet archive was used for cirrhosis detection with and without upstream liver segmentation. Classification performance for detection of liver cirrhosis was compared to two radiologists with different levels of experience (4^th-year resident, board-certified radiologist). Segmentation was performed using a U-Net architecture built on a pre-trained ResNet34 encoder. Differences in classification accuracy were assessed by the χ²-test.

Results

Dice coefficients for automatic segmentation were above 0.98 for both validation and test data. The classification accuracy of liver cirrhosis on validation (vACC) and test (tACC) data for the DTL pipeline with upstream liver segmentation (vACC = 0.99, tACC = 0.96) was significantly higher compared to the resident (vACC = 0.88, p < 0.01; tACC = 0.91, p = 0.01) and to the board-certified radiologist (vACC = 0.96, p < 0.01; tACC = 0.90, p < 0.01).

Conclusion

This proof-of-principle study demonstrates the potential of DTL for detecting cirrhosis based on standard T2-weighted MRI. The presented method for image-based diagnosis of liver cirrhosis demonstrated expert-level classification accuracy.

Key Points

• A pipeline consisting of two convolutional neural networks (CNNs) pre-trained on an extensive natural image database (ImageNet archive) enables detection of liver cirrhosis on standard T2-weighted MRI.

• High classification accuracy can be achieved even without altering the pre-trained parameters of the convolutional neural networks.

• Other abdominal structures apart from the liver were relevant for detection when the network was trained on unsegmented images.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00330-021-07858-1.

Radiomic and radiogenomic modeling for radiotherapy: strategies, pitfalls, and challenges

The power of predictive modeling for radiotherapy outcomes has historically been limited by an inability to adequately capture patient-specific variabilities; however, next-generation platforms together with imaging technologies and powerful bioinformatic tools have facilitated strategies and provided optimism. Integrating clinical, biological, imaging, and treatment-specific data for more accurate prediction of tumor control probabilities or risk of radiation-induced side effects are high-dimensional problems whose solutions could have widespread benefits to a diverse patient population-we discuss technical approaches toward this objective. Increasing interest in the above is specifically reflected by the emergence of two nascent fields, which are distinct but complementary: radiogenomics, which broadly seeks to integrate biological risk factors together with treatment and diagnostic information to generate individualized patient risk profiles, and radiomics, which further leverages large-scale imaging correlates and extracted features for the same purpose. We review classical analytical and data-driven approaches for outcomes prediction that serve as antecedents to both radiomic and radiogenomic strategies. Discussion then focuses on uses of conventional and deep machine learning in radiomics. We further consider promising strategies for the harmonization of high-dimensional, heterogeneous multiomics datasets (panomics) and techniques for nonparametric validation of best-fit models. Strategies to overcome common pitfalls that are unique to data-intensive radiomics are also discussed.