Current status and perspectives for computer-aided ultrasonic diagnosis of liver lesions using deep learning technology

Automated classification of multiple ophthalmic diseases using ultrasound images by deep learning

BACKGROUND

Ultrasound imaging is suitable for detecting and diagnosing ophthalmic abnormalities. However, a shortage of experienced sonographers and ophthalmologists remains a problem. This study aims to develop a multibranch transformer network (MBT-Net) for the automated classification of multiple ophthalmic diseases using B-mode ultrasound images.

METHODS

Ultrasound images with six clinically confirmed categories, including normal, retinal detachment, vitreous haemorrhage, intraocular tumour, posterior scleral staphyloma and other abnormalities, were used to develop and evaluate the MBT-Net. Images were derived from five different ultrasonic devices operated by different sonographers and divided into training set, validation set, internal testing set and temporal external testing set. Two senior ophthalmologists and two junior ophthalmologists were recruited to compare the model's performance.

RESULTS

A total of 10 184 ultrasound images were collected. The MBT-Net got an accuracy of 87.80% (95% CI 86.26% to 89.18%) in the internal testing set, which was significantly higher than junior ophthalmologists (95% CI 67.37% to 79.16%; both p<0.05) and lower than senior ophthalmologists (95% CI 89.45% to 92.61%; both p<0.05). The micro-average area under the curve of the six-category classification was 0.98. With reference to comprehensive clinical diagnosis, the measurements of agreement were almost perfect in the MBT-Net (kappa=0.85, p<0.05). There was no significant difference in the accuracy of the MBT-Net across five ultrasonic devices (p=0.27). The MBT-Net got an accuracy of 82.21% (95% CI 78.45% to 85.44%) in the temporal external testing set.

CONCLUSIONS

The MBT-Net showed high accuracy for screening and diagnosing multiple ophthalmic diseases using only ultrasound images across mutioperators and mutidevices.

Artifact reduction in photoacoustic images by generating virtual dense array sensor from hemispheric sparse array sensor using deep learning

PURPOSE

Vascular distribution is important information for diagnosing diseases and supporting surgery. Photoacoustic imaging is a technology that can image blood vessels noninvasively and with high resolution. In photoacoustic imaging, a hemispherical array sensor is especially suitable for measuring blood vessels running in various directions. However, as a hemispherical array sensor, a sparse array sensor is often used due to technical and cost issues, which causes artifacts in photoacoustic images. Therefore, in this study, we reduce these artifacts using deep learning technology to generate signals of virtual dense array sensors.

METHODS

Generating 2D virtual array sensor signals using a 3D convolutional neural network (CNN) requires huge computational costs and is impractical. Therefore, we installed virtual sensors between the real sensors along the spiral pattern in three different directions and used a 2D CNN to generate signals of the virtual sensors in each direction. Then we reconstructed a photoacoustic image using the signals from both the real sensors and the virtual sensors.

RESULTS

We evaluated the proposed method using simulation data and human palm measurement data. We found that these artifacts were significantly reduced in the images reconstructed using the proposed method, while the artifacts were strong in the images obtained only from the real sensor signals.

CONCLUSION

Using the proposed method, we were able to significantly reduce artifacts, and as a result, it became possible to recognize deep blood vessels. In addition, the processing time of the proposed method was sufficiently applicable to clinical measurement.

Artificial intelligence for the classification of focal liver lesions in ultrasound - a systematic review

Focal liver lesions are detected in about 15% of abdominal ultrasound examinations. The diagnosis of frequent benign lesions can be determined reliably based on the characteristic B-mode appearance of cysts, hemangiomas, or typical focal fatty changes. In the case of focal liver lesions which remain unclear on B-mode ultrasound, contrast-enhanced ultrasound (CEUS) increases diagnostic accuracy for the distinction between benign and malignant liver lesions. Artificial intelligence describes applications that try to emulate human intelligence, at least in subfields such as the classification of images. Since ultrasound is considered to be a particularly examiner-dependent technique, the application of artificial intelligence could be an interesting approach for an objective and accurate diagnosis. In this systematic review we analyzed how artificial intelligence can be used to classify the benign or malignant nature and entity of focal liver lesions on the basis of B-mode or CEUS data. In a structured search on Scopus, Web of Science, PubMed, and IEEE, we found 52 studies that met the inclusion criteria. Studies showed good diagnostic performance for both the classification as benign or malignant and the differentiation of individual tumor entities. The results could be improved by inclusion of clinical parameters and were comparable to those of experienced investigators in terms of diagnostic accuracy. However, due to the limited spectrum of lesions included in the studies and a lack of independent validation cohorts, the transfer of the results into clinical practice is limited.

A study on the optimal condition of ground truth area for liver tumor detection in ultrasound images using deep learning

PURPOSE

In recent years, efforts to apply artificial intelligence (AI) to the medical field have been growing. In general, a vast amount of high-quality training data is necessary to make great AI. For tumor detection AI, annotation quality is important. In diagnosis and detection of tumors using ultrasound images, humans use not only the tumor area but also the surrounding information, such as the back echo of the tumor. Therefore, we investigated changes in detection accuracy when changing the size of the region of interest (ROI, ground truth area) relative to liver tumors in the training data for the detection AI.

METHODS

We defined D/L as the ratio of the maximum diameter (D) of the liver tumor to the ROI size (L). We created training data by changing the D/L value, and performed learning and testing with YOLOv3.

RESULTS

Our results showed that the detection accuracy was highest when the training data were created with a D/L ratio between 0.8 and 1.0. In other words, it was found that the detection accuracy was improved by setting the ground true bounding box for detection AI training to be in contact with the tumor or slightly larger. We also found that when the D/L ratio was distributed in the training data, the wider the distribution, the lower the detection accuracy.

CONCLUSIONS

Therefore, we recommend that the detector be trained with the D/L value close to a certain value between 0.8 and 1.0 for liver tumor detection from ultrasound images.

Improving artificial intelligence pipeline for liver malignancy diagnosis using ultrasound images and video frames

Recent developments of deep learning methods have demonstrated their feasibility in liver malignancy diagnosis using ultrasound (US) images. However, most of these methods require manual selection and annotation of US images by radiologists, which limit their practical application. On the other hand, US videos provide more comprehensive morphological information about liver masses and their relationships with surrounding structures than US images, potentially leading to a more accurate diagnosis. Here, we developed a fully automated artificial intelligence (AI) pipeline to imitate the workflow of radiologists for detecting liver masses and diagnosing liver malignancy. In this pipeline, we designed an automated mass-guided strategy that used segmentation information to direct diagnostic models to focus on liver masses, thus increasing diagnostic accuracy. The diagnostic models based on US videos utilized bi-directional convolutional long short-term memory modules with an attention-boosted module to learn and fuse spatiotemporal information from consecutive video frames. Using a large-scale dataset of 50 063 US images and video frames from 11 468 patients, we developed and tested the AI pipeline and investigated its applications. A dataset of annotated US images is available at https://doi.org/10.5281/zenodo.7272660.

Artificial intelligence models for the diagnosis and management of liver diseases

With the development of more advanced methods for the diagnosis and treatment of diseases, the data required for medical care are becoming complex, and misinterpretation of information due to human error may result in serious consequences. Human error can be avoided with the support of artificial intelligence (AI). AI models trained with various medical data for diagnosis and management of liver diseases have been applied to hepatitis, fatty liver disease, liver cirrhosis, and liver cancer. Some of these models have been reported to outperform human experts in terms of performance, indicating their potential for supporting clinical practice given their high-speed output. This paper summarizes the recent advances in AI for liver disease and introduces the AI-aided diagnosis of liver tumors using B-mode ultrasonography.

Quantitative analysis of artificial intelligence on liver cancer: A bibliometric analysis

Objective

To provide the current research progress, hotspots, and emerging trends for AI in liver cancer, we have compiled a relative comprehensive and quantitative report on the research of liver disease using artificial intelligence by employing bibliometrics in this study.

Methods

In this study, the Web of Science Core Collection (WoSCC) database was used to perform systematic searches using keywords and a manual screening strategy, VOSviewer was used to analyze the degree of cooperation between countries/regions and institutions, as well as the co-occurrence of cooperation between authors and cited authors. Citespace was applied to generate a dual map to analyze the relationship of citing journals and citied journals and conduct a strong citation bursts ranking analysis of references. Online SRplot was used for in-depth keyword analysis and Microsoft Excel 2019 was used to collect the targeted variables from retrieved articles.

Results

1724 papers were collected in this study, including 1547 original articles and 177 reviews. The study of AI in liver cancer mostly began from 2003 and has developed rapidly from 2017. China has the largest number of publications, and the United States has the highest H-index and total citation counts. The top three most productive institutions are the League of European Research Universities, Sun Yat Sen University, and Zhejiang University. Jasjit S. Suri and Frontiers in Oncology are the most published author and journal, respectively. Keyword analysis showed that in addition to the research on liver cancer, research on liver cirrhosis, fatty liver disease, and liver fibrosis were also common. Computed tomography was the most used diagnostic tool, followed by ultrasound and magnetic resonance imaging. The diagnosis and differential diagnosis of liver cancer are currently the most widely adopted research goals, and comprehensive analyses of multi-type data and postoperative analysis of patients with advanced liver cancer are rare. The use of convolutional neural networks is the main technical method used in studies of AI on liver cancer.

Conclusion

AI has undergone rapid development and has a wide application in the diagnosis and treatment of liver diseases, especially in China. Imaging is an indispensable tool in this filed. Mmulti-type data fusion analysis and development of multimodal treatment plans for liver cancer could become the major trend of future research in AI in liver cancer.

The impact of low intensity ultrasound on cells: Underlying mechanisms and current status

Low intensity ultrasound (LIUS) has been adopted for a variety of therapeutic purposes because of its bioeffects such as thermal, mechanical, and cavitation effects. The mechanism of impact and cellular responses of LIUS in cellular regulations have been revealed, which helps to understand the role of LIUS in tumor treatment, stem cell therapy, and nervous system regulation. The review summarizes the bioeffects of LIUS at the cellular level and its related mechanisms, detailing the corresponding theoretical basis and latest research in the study of LIUS in the regulation of cells. In the future, the design of specific LIUS-mediated treatment strategies may benefit from promising investigations which is hoped to provide encouraging therapeutic data.

Research progress and hotspot of the artificial intelligence application in the ultrasound during 2011-2021: A bibliometric analysis

Ultrasound, as a common clinical examination tool, inevitably has human errors due to the limitations of manual operation. Artificial intelligence is an advanced computer program that can solve this problem. Therefore, the relevant literature on the application of artificial intelligence in the ultrasonic field from 2011 to 2021 was screened by authors from the Web of Science Core Collection, which aims to summarize the trend of artificial intelligence application in the field of ultrasound, meanwhile, visualize and predict research hotspots. A total of 908 publications were included in the study. Overall, the number of global publications is on the rise, and studies on the application of artificial intelligence in the field of ultrasound continue to increase. China has made the largest contribution in this field. In terms of institutions, Fudan University has the most number of publications. Recently, IEEE Access is the most published journal. Suri J. S. published most of the articles and had the highest number of citations in this field (29 articles). It's worth noting that, convolutional neural networks (CNN), as a kind of deep learning algorithm, was considered to bring better image analysis and processing ability in recent most-cited articles. According to the analysis of keywords, the latest keyword is "COVID-19" (2020.8). The co-occurrence analysis of keywords by VOSviewer visually presented four clusters which consisted of "deep learning," "machine learning," "application in the field of visceral organs," and "application in the field of cardiovascular". The latest hot words of these clusters were "COVID-19; neural-network; hepatocellular carcinoma; atherosclerotic plaques". This study reveals the importance of multi-institutional and multi-field collaboration in promoting research progress.

Deep learning in ultrasound elastography imaging: A review

It is known that changes in the mechanical properties of tissues are associated with the onset and progression of certain diseases. Ultrasound elastography is a technique to characterize tissue stiffness using ultrasound imaging either by measuring tissue strain using quasi-static elastography or natural organ pulsation elastography, or by tracing a propagated shear wave induced by a source or a natural vibration using dynamic elastography. In recent years, deep learning has begun to emerge in ultrasound elastography research. In this review, several common deep learning frameworks in the computer vision community, such as multilayer perceptron, convolutional neural network, and recurrent neural network are described. Then, recent advances in ultrasound elastography using such deep learning techniques are revisited in terms of algorithm development and clinical diagnosis. Finally, the current challenges and future developments of deep learning in ultrasound elastography are prospected. This article is protected by copyright. All rights reserved.

Application of endoscopic ultrasonography for detecting esophageal lesions based on convolutional neural network

BACKGROUND

A convolutional neural network (CNN) is a deep learning algorithm based on the principle of human brain visual cortex processing and image recognition.

AIM

To automatically identify the invasion depth and origin of esophageal lesions based on a CNN.

METHODS

A total of 1670 white-light images were used to train and validate the CNN system. The method proposed in this paper included the following two parts: (1) Location module, an object detection network, locating the classified main image feature regions of the image for subsequent classification tasks; and (2) Classification module, a traditional classification CNN, classifying the images cut out by the object detection network.

RESULTS

The CNN system proposed in this study achieved an overall accuracy of 82.49%, sensitivity of 80.23%, and specificity of 90.56%. In this study, after follow-up pathology, 726 patients were compared for endoscopic pathology. The misdiagnosis rate of endoscopic diagnosis in the lesion invasion range was approximately 9.5%; 41 patients showed no lesion invasion to the muscularis propria, but 36 of them pathologically showed invasion to the superficial muscularis propria. The patients with invasion of the tunica adventitia were all treated by surgery with an accuracy rate of 100%. For the examination of submucosal lesions, the accuracy of endoscopic ultrasonography (EUS) was approximately 99.3%. Results of this study showed that EUS had a high accuracy rate for the origin of submucosal lesions, whereas the misdiagnosis rate was slightly high in the evaluation of the invasion scope of lesions. Misdiagnosis could be due to different operating and diagnostic levels of endoscopists, unclear ultrasound probes, and unclear lesions.

CONCLUSION

This study is the first to recognize esophageal EUS images through deep learning, which can automatically identify the invasion depth and lesion origin of submucosal tumors and classify such tumors, thereby achieving good accuracy. In future studies, this method can provide guidance and help to clinical endoscopists.

Ultrasound of Fetal Cardiac Function Changes in Pregnancy-Induced Hypertension Syndrome

Pregnancy-induced hypertension syndrome (PIH) is a common pregnancy syndrome that could cause varying degrees of maternal and fetal organic damage and even endanger their lives. This study aimed to investigate ultrasound of fetal cardiac function changes in PIH. Totally 40 cases of gestational hypertension admitted to Cangzhou Central Hospital between October 2018 and September 2019 were enrolled in the hypertension group, and 40 women with healthy pregnancies during the same period were assigned to the normal group. Ultrasound results showed that PIH was associated with a significantly higher fetal cardiac septal thickness, fetal left ventricular end-diastolic area and end-systolic area, fetal right ventricular end-diastolic area and end-systolic area, fetal left ventricular systolic fraction 1 (VSF1), fetal left ventricular systolic fraction 2 (VSF2), fetal right VSF1, and fetal right VSF2 versus healthy pregnancy. The PIH fetuses had significantly lower neonatal weights versus healthy fetuses. Newborns of hypertensive pregnancies have larger hearts, faster heart rates, increased cardiac contractility, and lower weights versus newborns of healthy pregnancies.

Artificial intelligence (AI) models for the ultrasonographic diagnosis of liver tumors and comparison of diagnostic accuracies between AI and human experts

BACKGROUND

Ultrasonography (US) is widely used for the diagnosis of liver tumors. However, the accuracy of the diagnosis largely depends on the visual perception of humans. Hence, we aimed to construct artificial intelligence (AI) models for the diagnosis of liver tumors in US.

METHODS

We constructed three AI models based on still B-mode images: model-1 using 24,675 images, model-2 using 57,145 images, and model-3 using 70,950 images. A convolutional neural network was used to train the US images. The four-class liver tumor discrimination by AI, namely, cysts, hemangiomas, hepatocellular carcinoma, and metastatic tumors, was examined. The accuracy of the AI diagnosis was evaluated using tenfold cross-validation. The diagnostic performances of the AI models and human experts were also compared using an independent test cohort of video images.

RESULTS

The diagnostic accuracies of model-1, model-2, and model-3 in the four tumor types are 86.8%, 91.0%, and 91.1%, whereas those for malignant tumor are 91.3%, 94.3%, and 94.3%, respectively. In the independent comparison of the AIs and physicians, the percentages of correct diagnoses (accuracies) by the AIs are 80.0%, 81.8%, and 89.1% in model-1, model-2, and model-3, respectively. Meanwhile, the median percentages of correct diagnoses are 67.3% (range 63.6%-69.1%) and 47.3% (45.5%-47.3%) by human experts and non-experts, respectively.

CONCLUSION

The performance of the AI models surpassed that of human experts in the four-class discrimination and benign and malignant discrimination of liver tumors. Thus, the AI models can help prevent human errors in US diagnosis.

A New Dawn for the Use of Artificial Intelligence in Gastroenterology, Hepatology and Pancreatology

Artificial intelligence (AI) is rapidly becoming an essential tool in the medical field as well as in daily life. Recent developments in deep learning, a subfield of AI, have brought remarkable advances in image recognition, which facilitates improvement in the early detection of cancer by endoscopy, ultrasonography, and computed tomography. In addition, AI-assisted big data analysis represents a great step forward for precision medicine. This review provides an overview of AI technology, particularly for gastroenterology, hepatology, and pancreatology, to help clinicians utilize AI in the near future.

The overview of the deep learning integrated into the medical imaging of liver: a review

Deep learning (DL) is a recently developed artificial intelligent method that can be integrated into numerous fields. For the imaging diagnosis of liver disease, several remarkable outcomes have been achieved with the application of DL currently. This advanced algorithm takes part in various sections of imaging processing such as liver segmentation, lesion delineation, disease classification, process optimization, etc. The DL optimized imaging diagnosis shows a broad prospect instead of the pathological biopsy for the advantages of convenience, safety, and inexpensiveness. In this paper, we reviewed the published representative DL-related hepatic imaging works, described the general situation of this new-rising technology in medical liver imaging and explored the future direction of DL development.

Development and validation of artificial intelligence to detect and diagnose liver lesions from ultrasound images

Artificial intelligence (AI) using a convolutional neural network (CNN) has demonstrated promising performance in radiological analysis. We aimed to develop and validate a CNN for the detection and diagnosis of focal liver lesions (FLLs) from ultrasonography (USG) still images. The CNN was developed with a supervised training method using 40,397 retrospectively collected images from 3,487 patients, including 20,432 FLLs (hepatocellular carcinomas (HCCs), cysts, hemangiomas, focal fatty sparing, and focal fatty infiltration). AI performance was evaluated using an internal test set of 6,191 images with 845 FLLs, then externally validated using 18,922 images with 1,195 FLLs from two additional hospitals. The internal evaluation yielded an overall detection rate, diagnostic sensitivity and specificity of 87.0% (95%CI: 84.3-89.6), 83.9% (95%CI: 80.3-87.4), and 97.1% (95%CI: 96.5-97.7), respectively. The CNN also performed consistently well on external validation cohorts, with a detection rate, diagnostic sensitivity and specificity of 75.0% (95%CI: 71.7-78.3), 84.9% (95%CI: 81.6-88.2), and 97.1% (95%CI: 96.5-97.6), respectively. For diagnosis of HCC, the CNN yielded sensitivity, specificity, and negative predictive value (NPV) of 73.6% (95%CI: 64.3-82.8), 97.8% (95%CI: 96.7-98.9), and 96.5% (95%CI: 95.0-97.9) on the internal test set; and 81.5% (95%CI: 74.2-88.8), 94.4% (95%CI: 92.8-96.0), and 97.4% (95%CI: 96.2-98.5) on the external validation set, respectively. CNN detected and diagnosed common FLLs in USG images with excellent specificity and NPV for HCC. Further development of an AI system for real-time detection and characterization of FLLs in USG is warranted.

Artificial Intelligence in Medical Imaging and Its Application in Sonography for the Management of Liver Tumor

Recent advancement in artificial intelligence (AI) facilitate the development of AI-powered medical imaging including ultrasonography (US). However, overlooking or misdiagnosis of malignant lesions may result in serious consequences; the introduction of AI to the imaging modalities may be an ideal solution to prevent human error. For the development of AI for medical imaging, it is necessary to understand the characteristics of modalities on the context of task setting, required data sets, suitable AI algorism, and expected performance with clinical impact. Regarding the AI-aided US diagnosis, several attempts have been made to construct an image database and develop an AI-aided diagnosis system in the field of oncology. Regarding the diagnosis of liver tumors using US images, 4- or 5-class classifications, including the discrimination of hepatocellular carcinoma (HCC), metastatic tumors, hemangiomas, liver cysts, and focal nodular hyperplasia, have been reported using AI. Combination of radiomic approach with AI is also becoming a powerful tool for predicting the outcome in patients with HCC after treatment, indicating the potential of AI for applying personalized medical care. However, US images show high heterogeneity because of differences in conditions during the examination, and a variety of imaging parameters may affect the quality of images; such conditions may hamper the development of US-based AI. In this review, we summarized the development of AI in medical images with challenges to task setting, data curation, and focus on the application of AI for the managements of liver tumor, especially for US diagnosis.