Real-Time Automatic Artery Segmentation, Reconstruction and Registration for Ultrasound-Guided Regional Anaesthesia of the Femoral Nerve

Artificial intelligence for ultrasound scanning in regional anaesthesia: a scoping review of the evidence from multiple disciplines

BACKGROUND

Artificial intelligence (AI) for ultrasound scanning in regional anaesthesia is a rapidly developing interdisciplinary field. There is a risk that work could be undertaken in parallel by different elements of the community but with a lack of knowledge transfer between disciplines, leading to repetition and diverging methodologies. This scoping review aimed to identify and map the available literature on the accuracy and utility of AI systems for ultrasound scanning in regional anaesthesia.

METHODS

A literature search was conducted using Medline, Embase, CINAHL, IEEE Xplore, and ACM Digital Library. Clinical trial registries, a registry of doctoral theses, regulatory authority databases, and websites of learned societies in the field were searched. Online commercial sources were also reviewed.

RESULTS

In total, 13,014 sources were identified; 116 were included for full-text review. A marked change in AI techniques was noted in 2016-17, from which point on the predominant technique used was deep learning. Methods of evaluating accuracy are variable, meaning it is impossible to compare the performance of one model with another. Evaluations of utility are more comparable, but predominantly gained from the simulation setting with limited clinical data on efficacy or safety. Study methodology and reporting lack standardisation.

CONCLUSIONS

There is a lack of structure to the evaluation of accuracy and utility of AI for ultrasound scanning in regional anaesthesia, which hinders rigorous appraisal and clinical uptake. A framework for consistent evaluation is needed to inform model evaluation, allow comparison between approaches/models, and facilitate appropriate clinical adoption.

The Big Bang of Deep Learning in Ultrasound-Guided Surgery: A Review

Ultrasound (US) imaging is a paramount modality in many image-guided surgeries and percutaneous interventions, thanks to its high portability, temporal resolution, and cost-efficiency. However, due to its imaging principles, the US is often noisy and difficult to interpret. Appropriate image processing can greatly enhance the applicability of the imaging modality in clinical practice. Compared with the classic iterative optimization and machine learning (ML) approach, deep learning (DL) algorithms have shown great performance in terms of accuracy and efficiency for US processing. In this work, we conduct a comprehensive review on deep-learning algorithms in the applications of US-guided interventions, summarize the current trends, and suggest future directions on the topic.

Unsupervised domain adaptation method for segmenting cross-sectional CCA images

BACKGROUND AND OBJECTIVES

Automatic vessel segmentation in ultrasound is challenging due to the quality of the ultrasound images, which is affected by attenuation, high level of speckle noise and acoustic shadowing. Recently, deep convolutional neural networks are increasing in popularity due to their great performance on image segmentation problems, including vessel segmentation. Traditionally, large labeled datasets are required to train a network that achieves high performance, and is able to generalize well to different orientations, transducers and ultrasound scanners. However, these large datasets are rare, given that it is challenging and time-consuming to acquire and manually annotate in-vivo data.

METHODS

In this work, we present a model-based, unsupervised domain adaptation method that consists of two stages. In the first stage, the network is trained on simulated ultrasound images, which have an accurate ground truth. In the second stage, the network continues training on in-vivo data in an unsupervised way, therefore not requiring the data to be labelled. Rather than using an adversarial neural network, prior knowledge on the elliptical shape of the segmentation mask is used to detect unexpected outputs.

RESULTS

The segmentation performance was quantified using manually segmented images as ground truth. Due to the proposed domain adaptation method, the median Dice similarity coefficient increased from 0 to 0.951, outperforming a domain adversarial neural network (median Dice 0.922) and a state-of-the-art Star-Kalman algorithm that was specifically designed for this dataset (median Dice 0.942).

CONCLUSIONS

The results show that it is feasible to first train a neural network on simulated data, and then apply model-based domain adaptation to further improve segmentation performance by training on unlabeled in-vivo data. This overcomes the limitation of conventional deep learning approaches to require large amounts of manually labeled in-vivo data. Since the proposed domain adaptation method only requires prior knowledge on the shape of the segmentation mask, performance can be explored in various domains and applications in future research.

A real-time freehand 3D ultrasound imaging method for scoliosis assessment

Real‐time 3D ultrasound has gained popularity in many fields because it can provide interactive feedback to help acquire high‐quality images or to conduct timely diagnosis. However, no comprehensive study has been reported on such an imaging method for scoliosis evaluation due to the complexity of this application. Meanwhile, the use of radiation‐free assessment of scoliosis is becoming increasingly popular. This study developed a real‐time 3D ultrasound imaging method for scoliosis assessment based on an incremental imaging method. In vivo experiments involving 36 patients with scoliosis were performed to test the performance of the proposed method. This new imaging method achieved a mean incremental frame rate of 82.7 ± 11.0 frames/s. The high repeatability of the intra‐operator test (intraclass correlation coefficient [ICC] = 0.92) and inter‐operator test (ICC = 0.91) demonstrated that the new method was very reliable. The result of spinous process angles obtained by the new method was linearly correlated (y = 0.97x, R² = 0.88) with that obtained by conventional 3D reconstruction. These results suggested that the newly developed imaging method can provide real‐time ultrasound imaging for scoliosis evaluation while preserving the comparative image quality of the conventional 3D reconstruction method.

Artificial Intelligence: Innovation to Assist in the Identification of Sono-anatomy for Ultrasound-Guided Regional Anaesthesia

Ultrasound-guided regional anaesthesia (UGRA) involves the targeted deposition of local anaesthesia to inhibit the function of peripheral nerves. Ultrasound allows the visualisation of nerves and the surrounding structures, to guide needle insertion to a perineural or fascial plane end point for injection. However, it is challenging to develop the necessary skills to acquire and interpret optimal ultrasound images. Sound anatomical knowledge is required and human image analysis is fallible, limited by heuristic behaviours and fatigue, while its subjectivity leads to varied interpretation even amongst experts. Therefore, to maximise the potential benefit of ultrasound guidance, innovation in sono-anatomical identification is required.Artificial intelligence (AI) is rapidly infiltrating many aspects of everyday life. Advances related to medicine have been slower, in part because of the regulatory approval process needing to thoroughly evaluate the risk-benefit ratio of new devices. One area of AI to show significant promise is computer vision (a branch of AI dealing with how computers interpret the visual world), which is particularly relevant to medical image interpretation. AI includes the subfields of machine learning and deep learning, techniques used to interpret or label images. Deep learning systems may hold potential to support ultrasound image interpretation in UGRA but must be trained and validated on data prior to clinical use.Review of the current UGRA literature compares the success and generalisability of deep learning and non-deep learning approaches to image segmentation and explains how computers are able to track structures such as nerves through image frames. We conclude this review with a case study from industry (ScanNav Anatomy Peripheral Nerve Block; Intelligent Ultrasound Limited). This includes a more detailed discussion of the AI approach involved in this system and reviews current evidence of the system performance.The authors discuss how this technology may be best used to assist anaesthetists and what effects this may have on the future of learning and practice of UGRA. Finally, we discuss possible avenues for AI within UGRA and the associated implications.

A Generalized Approach for Automatic 3-D Geometry Assessment of Blood Vessels in Transverse Ultrasound Images Using Convolutional Neural Networks

Accurate 3-D geometries of arteries and veins are important clinical data for diagnosis of arterial disease and intervention planning. Automatic segmentation of vessels in the transverse view suffers from the low lateral resolution and contrast. Convolutional neural networks are a promising tool for automatic segmentation of medical images, outperforming the traditional segmentation methods with high robustness. In this study, we aim to create a general, robust, and accurate method to segment the lumen-wall boundary of healthy central and peripheral vessels in large field-of-view freehand ultrasound (US) datasets. Data were acquired using the freehand US, in combination with a probe tracker. A total of ±36 000 cross-sectional images, acquired in the common, internal, and external carotid artery ( N = 37 ), in the radial, ulnar artery, and cephalic vein ( N = 12 ), and in the femoral artery ( N = 5 ) were included. To create masks (of the lumen) for training data, a conventional automatic segmentation method was used. The neural networks were trained on: 1) data of all vessels and 2) the carotid artery only. The performance was compared and tested using an open-access dataset. The recall, precision, DICE, and intersection over union (IoU) were calculated. Overall, segmentation was successful in the carotid and peripheral arteries. The Multires U-net architecture performs best overall with DICE = 0.93 when trained on the total dataset. Future studies will focus on the inclusion of vascular pathologies.

A CNN Segmentation-Based Approach to Object Detection and Tracking in Ultrasound Scans with Application to the Vagus Nerve Detection

Ultrasound scanning is essential in several medical diagnostic and therapeutic applications. It is used to visualize and analyze anatomical features and structures that influence treatment plans. However, it is both labor intensive, and its effectiveness is operator dependent. Real-time accurate and robust automatic detection and tracking of anatomical structures while scanning would significantly impact diagnostic and therapeutic procedures to be consistent and efficient. In this paper, we propose a deep learning framework to automatically detect and track a specific anatomical target structure in ultrasound scans. Our framework is designed to be accurate and robust across subjects and imaging devices, to operate in real-time, and to not require a large training set. It maintains a localization precision and recall higher than 90% when trained on training sets that are as small as 20% in size of the original training set. The framework backbone is a weakly trained segmentation neural network based on U-Net. We tested the framework on two different ultrasound datasets with the aim to detect and track the Vagus nerve, where it outperformed current state-of-the-art real-time object detection networks.Clinical Relevance-The proposed approach provides an accurate method to detect and localize target anatomical structures in real-time, assisting sonographers during ultrasound scanning sessions by reducing diagnostic and detection errors, and expediting the duration of scanning sessions.

Shape constraint function for artery tracking in ultrasound images

Ultrasound guided regional anesthesia (UGRA) has emerged as a powerful technique for pain management in the operating theatre. It uses ultrasound imaging to visualize anatomical structures, the needle insertion and the delivery of the anesthetic around the targeted nerve block. Detection of the nerves is a difficult task, however, due to the poor quality of the ultrasound images. Recent developments in pattern recognition and machine learning have heightened the need for computer aided systems in many applications. This type of system can improve UGRA practice. In many imaging situations nerves are not salient in images. Generally, practitioners rely on the arteries as key anatomical structures to confirm the positions of the nerves, making artery tracking an important aspect for UGRA procedure. However, artery tracking in a noisy environment is a challenging problem, due to the instability of the features. This paper proposes a new method for real-time artery tracking in ultrasound images. It is based on shape information to correct tracker location errors. A new objective function is proposed, which defines an artery as an elliptical shape, enabling its robust fitting in a noisy environment. This approach is incorporated in two well-known tracking algorithms, and shows a systematic improvement over the original trackers. Evaluations were performed on 71 videos of different axillary nerve blocks. The results obtained demonstrated the validity of the proposed method.

Vessel segmentation for automatic registration of untracked laparoscopic ultrasound to CT of the liver

Purpose:

Registration of Laparoscopic Ultrasound (LUS) to a pre-operative scan such as Computed Tomography (CT) using blood vessel information has been proposed as a method to enable image-guidance for laparoscopic liver resection. Currently, there are solutions for this problem that can potentially enable clinical translation by bypassing the need for a manual initialisation and tracking information. However, no reliable framework for the segmentation of vessels in 2D untracked LUS images has been presented.

Methods:

We propose the use of 2D UNet for the segmentation of liver vessels in 2D LUS images. We integrate these results in a previously developed registration method, and show the feasibility of a fully automatic initialisation to the LUS to CT registration problem without a tracking device.

Results:

We validate our segmentation using LUS data from 6 patients. We test multiple models by placing patient datasets into different combinations of training, testing and hold-out, and obtain mean Dice scores ranging from 0.543 to 0.706. Using these segmentations, we obtain registration accuracies between 6.3 and 16.6 mm in 50% of cases.

Conclusions:

We demonstrate the first instance of deep learning (DL) for the segmentation of liver vessels in LUS. Our results show the feasibility of UNet in detecting multiple vessel instances in 2D LUS images, and potentially automating a LUS to CT registration pipeline.

Mechanical characterization of abdominal aortas using multi-perspective ultrasound imaging

Mechanical characterization of abdominal aortic aneurysms using personalized biomechanical models is being widely investigated as an alternative criterion to assess risk of rupture. These methods rely on accurate wall motion detection and appropriate model boundary conditions. In this study, multi-perspective ultrasound is combined with finite element models to perform mechanical characterization of abdominal aortas in volunteers. Multi-perspective biplane radio frequency ultrasound recordings were made under seven angles (-45° to 45°) in one phantom set-up and eight volunteers, which were merged using automatic image registration. 2-D displacement fields were estimated in the seven longitudinal ultrasound views, creating a sparse, high resolution 3-D map of the wall motion at relatively high frame rates (20-27 Hz). The displacements were used to personalize the subject-specific finite element model of which the geometry of the aorta, spine, and surrounding tissue were determined from a single 3-D ultrasound acquisition. Automatic registration of the multi-perspective images was successful in six out of eight cases with an average error of 5.4° compared to the ground truth. Displacements of the aortic wall were measured and cyclic strain of the aortic diameter was found ranging from 4.2% to 8.6%. The subject-specific mesh and inverse FE analysis was performed yielding shear moduli estimates for the wall between 104 and 215 kPa. Comparative results from a single-perspective workflow revealed very low aortic wall motion signal, which resulted in relatively high modulus estimates, between 230 and 754 kPa. Multi-perspective biplane ultrasound imaging was used to personalize finite element models of the abdominal aorta and its surroundings, and performing mechanical characterization of the aortic shear modulus. The method was found to be a more robust method compared to a single-perspective 3-D ultrasound approach. Future research will focus on investigating the use of multiple 3-D ultrasound acquisitions, the feasibility of free-hand scanning, the creation of a full 3-D automatic registration process, and with that, enable a clinical continuation of this study.

Enhancing Lateral Contrast Using Multi-perspective Ultrasound Imaging of Abdominal Aortas

Vascular ultrasound imaging is inherently hampered by low lateral resolution and contrast. Steering of the ultrasound beams can be used to overcome these limitations in superficial artery imaging because the aperture-to-depth ratio is relatively high. However, in arteries located at larger depths, the steered beams do not overlap for larger steering angles. In this study, the ultrasound probe is physically translated over the abdomen to create large angles between acquisitions, while maintaining overlap on the abdominal aorta. In one phantom setup and 11 volunteers, 2-D cross-sectional multi-perspective ultrasound images of the abdominal aorta were acquired using seven angles between -45° and +45°. Automatic registration of the recorded images was performed by automatic feature detection of the aorta and spine. This automatic detection was successful in 62 out of 77 image sets. Compounded multi-perspective images showed an increase of contrast-to-noise ratios from 0.6 ± 0.1 to 1.2 ± 0.2 over the entire heart cycle in volunteers.

An Angle-Independent Cross-Sectional Doppler Method for Flow Estimation in the Common Carotid Artery

The Doppler ultrasound is the most common technique for noninvasive quantification of blood flow, which, in turn, is of major clinical importance for the assessment of the cardiovascular condition. In this article, a method is proposed in which the vessel is imaged in the short axis, which has the advantage of capturing the whole flow profile while measuring the vessel area simultaneously. This view is easier to obtain than the longitudinal image that is currently used in flow velocity estimation, reducing operator dependence. However, the Doppler angle in cross-sectional images is unknown since the vessel wall can no longer be used to estimate the flow direction. The proposed method to estimate the Doppler angle in these images is based on the elliptical intersection between a cylindrical vessel and the ultrasound plane. The parameters of this ellipse (major axis, minor axis, and rotation) are used to estimate the Doppler angle by solving a least-squares problem. Theoretical feasibility was shown in a geometrical model, after which the Doppler angle was estimated in simulated ultrasound images generated in Field II, yielding a mean error within 4°. In vitro, across 15 short-axis measurements with a wide variety of Doppler angles, errors in the flow estimates were below 10%, and in vivo, the average velocities in systole obtained from longitudinal ( v=69.1 cm/s) and cross-sectional ( v=66.5 cm/s) acquisitions were in agreement. Further research is required to validate these results on a larger population.

Automated 3D geometry segmentation of the healthy and diseased carotid artery in free-hand, probe tracked ultrasound images

PURPOSE

Rupture of an arterosclerotic plaque in the carotid artery is a major cause of stroke. Biomechanical analysis of plaques is under development aiming to aid the clinician in the assessment of plaque vulnerability. Patient-specific three-dimensional (3D) geometry assessment of the carotid artery, including the bifurcation, is required as input for these biomechanical models. This requires a high-resolution, 3D, noninvasive imaging modality such as ultrasound (US). In this study, a high-resolution two-dimensional (2D) linear array in combination with a magnetic probe tracking device and automatic segmentation method was used to assess the geometry of the carotid artery. The advantages of using this system over a 3D ultrasound probe are its higher resolution (spatial and temporal) and its larger field of view.

METHODS

A slow sweep (v = ± 5 mm/s) was made over the subject's neck so that the full geometry of the bifurcated geometry of the carotid artery is captured. An automated segmentation pipeline was developed. First, the Star-Kalman method was used to approximate the center and size of the vessels for every frame. Images were filtered with a Gaussian high-pass filter before conversion into the 2D monogenic signals, and multiscale asymmetry features were extracted from these data, enhancing low lateral wall-lumen contrast. These images, in combination with the initial ellipse contours, were used for an active deformable contour model to segment the vessel lumen. To segment the lumen-plaque boundary, Otsu's automatic thresholding method was used. Distension of the wall due to the change in blood pressure was removed using a filter approach. Finally, the contours were converted into a 3D hexahedral mesh for a patient-specific solid mechanics model of the complete arterial wall.

RESULTS

The method was tested on 19 healthy volunteers and on 3 patients. The results were compared to manual segmentation performed by three experienced observers. Results showed an average Hausdorff distance of 0.86 mm and an average similarity index of 0.91 for the common carotid artery (CCA) and 0.88 for the internal and external carotid artery. For the total algorithm, the success rate was 89%, in 4 out of 38 datasets the ICA and ECA were not sufficient visible in the US images. Accurate 3D hexahedral meshes were successfully generated from the segmented images .

CONCLUSIONS

With this method, a subject-specific biomechanical model can be constructed directly from a hand-held 2D US measurement, within 10 min, with a minimal user input. The performance of the proposed segmentation algorithm is comparable to or better than algorithms previously described in literature. Moreover, the algorithm is able to segment the CCA, ICA, and ECA including the carotid bifurcation in transverse B-mode images in both healthy and diseased arteries.

Highlighting nerves and blood vessels for ultrasound-guided axillary nerve block procedures using neural networks

Ultrasound images acquired during axillary nerve block procedures can be difficult to interpret. Highlighting the important structures, such as nerves and blood vessels, may be useful for the training of inexperienced users. A deep convolutional neural network is used to identify the musculocutaneous, median, ulnar, and radial nerves, as well as the blood vessels in ultrasound images. A dataset of 49 subjects is collected and used for training and evaluation of the neural network. Several image augmentations, such as rotation, elastic deformation, shadows, and horizontal flipping, are tested. The neural network is evaluated using cross validation. The results showed that the blood vessels were the easiest to detect with a precision and recall above 0.8. Among the nerves, the median and ulnar nerves were the easiest to detect with an F -score of 0.73 and 0.62, respectively. The radial nerve was the hardest to detect with an F -score of 0.39. Image augmentations proved effective, increasing F -score by as much as 0.13. A Wilcoxon signed-rank test showed that the improvement from rotation, shadow, and elastic deformation augmentations were significant and the combination of all augmentations gave the best result. The results are promising; however, there is more work to be done, as the precision and recall are still too low. A larger dataset is most likely needed to improve accuracy, in combination with anatomical and temporal models.

A Review on Real-Time 3D Ultrasound Imaging Technology

Real-time three-dimensional (3D) ultrasound (US) has attracted much more attention in medical researches because it provides interactive feedback to help clinicians acquire high-quality images as well as timely spatial information of the scanned area and hence is necessary in intraoperative ultrasound examinations. Plenty of publications have been declared to complete the real-time or near real-time visualization of 3D ultrasound using volumetric probes or the routinely used two-dimensional (2D) probes. So far, a review on how to design an interactive system with appropriate processing algorithms remains missing, resulting in the lack of systematic understanding of the relevant technology. In this article, previous and the latest work on designing a real-time or near real-time 3D ultrasound imaging system are reviewed. Specifically, the data acquisition techniques, reconstruction algorithms, volume rendering methods, and clinical applications are presented. Moreover, the advantages and disadvantages of state-of-the-art approaches are discussed in detail.

Automatic Segmentation and Probe Guidance for Real-Time Assistance of Ultrasound-Guided Femoral Nerve Blocks

Ultrasound-guided regional anesthesia can be challenging, especially for inexperienced physicians. The goal of the proposed methods is to create a system that can assist a user in performing ultrasound-guided femoral nerve blocks. The system indicates in which direction the user should move the ultrasound probe to investigate the region of interest and to reach the target site for needle insertion. Additionally, the system provides automatic real-time segmentation of the femoral artery, the femoral nerve and the two layers fascia lata and fascia iliaca. This aids in interpretation of the 2-D ultrasound images and the surrounding anatomy in 3-D. The system was evaluated on 24 ultrasound acquisitions of both legs from six subjects. The estimated target site for needle insertion and the segmentations were compared with those of an expert anesthesiologist. Average target distance was 8.5 mm with a standard deviation of 2.5 mm. The mean absolute differences of the femoral nerve and the fascia segmentations were about 1-3 mm.