2D ultrasound imaging based intra-fraction respiratory motion tracking for abdominal radiation therapy using machine learning

Artificial intelligence in abdominal and pelvic ultrasound imaging: current applications

BACKGROUND

In recent years, the integration of artificial intelligence (AI) techniques into medical imaging has shown great potential to transform the diagnostic process. This review aims to provide a comprehensive overview of current state-of-the-art applications for AI in abdominal and pelvic ultrasound imaging.

METHODS

We searched the PubMed, FDA, and ClinicalTrials.gov databases for applications of AI in abdominal and pelvic ultrasound imaging.

RESULTS

A total of 128 titles were identified from the database search and were eligible for screening. After screening, 57 manuscripts were included in the final review. The main anatomical applications included multi-organ detection (n = 16, 28%), gynecology (n = 15, 26%), hepatobiliary system (n = 13, 23%), and musculoskeletal (n = 8, 14%). The main methodological applications included deep learning (n = 37, 65%), machine learning (n = 13, 23%), natural language processing (n = 5, 9%), and robots (n = 2, 4%). The majority of the studies were single-center (n = 43, 75%) and retrospective (n = 56, 98%). We identified 17 FDA approved AI ultrasound devices, with only a few being specifically used for abdominal/pelvic imaging (infertility monitoring and follicle development).

CONCLUSION

The application of AI in abdominal/pelvic ultrasound shows promising early results for disease diagnosis, monitoring, and report refinement. However, the risk of bias remains high because very few of these applications have been prospectively validated (in multi-center studies) or have received FDA clearance.

Artificial intelligence-based motion tracking in cancer radiotherapy: A review

Radiotherapy aims to deliver a prescribed dose to the tumor while sparing neighboring organs at risk (OARs). Increasingly complex treatment techniques such as volumetric modulated arc therapy (VMAT), stereotactic radiosurgery (SRS), stereotactic body radiotherapy (SBRT), and proton therapy have been developed to deliver doses more precisely to the target. While such technologies have improved dose delivery, the implementation of intra-fraction motion management to verify tumor position at the time of treatment has become increasingly relevant. Artificial intelligence (AI) has recently demonstrated great potential for real-time tracking of tumors during treatment. However, AI-based motion management faces several challenges, including bias in training data, poor transparency, difficult data collection, complex workflows and quality assurance, and limited sample sizes. This review presents the AI algorithms used for chest, abdomen, and pelvic tumor motion management/tracking for radiotherapy and provides a literature summary on the topic. We will also discuss the limitations of these AI-based studies and propose potential improvements.

Enhancing Image-Guided Radiation Therapy for Pancreatic Cancer: Utilizing Aligned Peak Response Beamforming in Flexible Array Transducers

To develop ultrasound-guided radiotherapy, we proposed an assistant structure with embedded markers along with a novel alternative method, the Aligned Peak Response (APR) method, to alter the conventional delay-and-sum (DAS) beamformer for reconstructing ultrasound images obtained from a flexible array. We simulated imaging targets in Field-II using point target phantoms with point targets at different locations. In the experimental phantom ultrasound images, image RF data were acquired with a flexible transducer with in-house assistant structures embedded with needle targets for testing the accuracy of the APR method. The lateral full width at half maximum (FWHM) values of the objective point target (OPT) in ground truth ultrasound images, APR-delayed ultrasound images with a flat shape, and images acquired with curved transducer radii of 500 mm and 700 mm were 3.96 mm, 4.95 mm, 4.96 mm, and 4.95 mm. The corresponding axial FWHM values were 1.52 mm, 4.08 mm, 5.84 mm, and 5.92 mm, respectively. These results demonstrate that the proposed assistant structure and the APR method have the potential to construct accurate delay curves without external shape sensing, thereby enabling a flexible ultrasound array for tracking pancreatic tumor targets in real time for radiotherapy.

DAART: a deep learning platform for deeply accelerated adaptive radiation therapy for lung cancer

Purpose

The study aimed to implement a novel, deeply accelerated adaptive radiation therapy (DAART) approach for lung cancer radiotherapy (RT). Lung cancer is the most common cause of cancer-related death, and RT is the preferred medically inoperable treatment for early stage non-small cell lung cancer (NSCLC). In the current lengthy workflow, it takes a median of four weeks from diagnosis to RT treatment, which can result in complete restaging and loss of local control with delay. We implemented the DAART approach, featuring a novel deepPERFECT system, to address unwanted delays between diagnosis and treatment initiation.

Materials and methods

We developed a deepPERFECT to adapt the initial diagnostic imaging to the treatment setup to allow initial RT planning and verification. We used data from 15 patients with NSCLC treated with RT to train the model and test its performance. We conducted a virtual clinical trial to evaluate the treatment quality of the proposed DAART for lung cancer radiotherapy.

Results

We found that deepPERFECT predicts planning CT with a mean high-intensity fidelity of 83 and 14 HU for the body and lungs, respectively. The shape of the body and lungs on the synthesized CT was highly conformal, with a dice similarity coefficient (DSC) of 0.91, 0.97, and Hausdorff distance (HD) of 7.9 mm, and 4.9 mm, respectively, compared with the planning CT scan. The tumor showed less conformality, which warrants acquisition of treatment Day1 CT and online adaptive RT. An initial plan was designed on synthesized CT and then adapted to treatment Day1 CT using the adapt to position (ATP) and adapt to shape (ATS) method. Non-inferior plan quality was achieved by the ATP scenario, while all ATS-adapted plans showed good plan quality.

Conclusion

DAART reduces the common online ART (ART) treatment course by at least two weeks, resulting in a 50% shorter time to treatment to lower the chance of restaging and loss of local control.

Ultrasound Imaging with Flexible Array Transducer for Pancreatic Cancer Radiation Therapy

Pancreatic cancer with less than 10% 3-year survival rate is one of deadliest cancer types and greatly benefits from enhanced radiotherapy. Organ motion monitoring helps spare the normal tissue from high radiation and, in turn, enables the dose escalation to the target that has been shown to improve the effectiveness of RT by doubling and tripling post-RT survival rate. The flexible array transducer is a novel and promising solution to address the limitation of conventional US probes. We proposed a novel shape estimation for flexible array transducer using two sequential algorithms: (i) an optical tracking-based system that uses the optical markers coordinates attached to the probe at specific positions to estimate the array shape in real-time and (ii) a fully automatic shape optimization algorithm that automatically searches for the optimal array shape that results in the highest quality reconstructed image. We conducted phantom and in vivo experiments to evaluate the estimated array shapes and the accuracy of reconstructed US images. The proposed method reconstructed US images with low full-width-at-half-maximum (FWHM) of the point scatters, correct aspect ratio of the cyst, and high-matching score with the ground truth. Our results demonstrated that the proposed methods reconstruct high-quality ultrasound images with significantly less defocusing and distortion compared with those without any correction. Specifically, the automatic optimization method reduced the array shape estimation error to less than half-wavelength of transmitted wave, resulting in a high-quality reconstructed image.

Landmark tracking in liver US images using cascade convolutional neural networks with long short-term memory

Accurate tracking of anatomic landmarks is critical for motion management in liver radiation therapy. Ultrasound (US) is a safe, low-cost technology that is broadly available and offer real-time imaging capability. This study proposed a deep learning-based tracking method for the US image-guided radiation therapy. The proposed cascade deep learning model is composed of an attention network, a mask region-based convolutional neural network (mask R-CNN), and a long short-term memory (LSTM) network. The attention network learns a mapping from an US image to a suspected area of landmark motion in order to reduce the search region. The mask R-CNN then produces multiple region-of-interest proposals in the reduced region and identifies the proposed landmark via three network heads: bounding box regression, proposal classification, and landmark segmentation. The LSTM network models the temporal relationship among the successive image frames for bounding box regression and proposal classification. To consolidate the final proposal, a selection method is designed according to the similarities between sequential frames. The proposed method was tested on the liver US tracking datasets used in the medical image computing and computer assisted interventions 2015 challenges, where the landmarks were annotated by three experienced observers to obtain their mean positions. Five-fold cross validation on the 24 given US sequences with ground truths shows that the mean tracking error for all landmarks is 0.65 ± 0.56 mm, and the errors of all landmarks are within 2 mm. We further tested the proposed model on 69 landmarks from the testing dataset that have the similar image pattern with the training pattern, resulting in a mean tracking error of 0.94 ± 0.83 mm. The proposed deep-learning model was implemented on a graphics processing unit (GPU), tracking 47-81 frames s-1. Our experimental results have demonstrated the feasibility and accuracy of our proposed method in tracking liver anatomic landmarks using US images, providing a potential solution for real-time liver tracking for active motion management during radiation therapy.

Assessment of experimental OpenCV tracking algorithms for ultrasound videos

This study aims to compare the tracking algorithms provided by the OpenCV library to use on ultrasound video. Despite the widespread application of this computer vision library, few works describe the attempts to use it to track the movement of liver tumors on ultrasound video. Movements of the neoplasms caused by the patient`s breath interfere with the positioning of the instruments during the process of biopsy and radio-frequency ablation. The main hypothesis of the experiment was that tracking neoplasms and correcting the position of the manipulator in case of using robotic-assisted surgery will allow positioning the instruments more precisely. Another goal of the experiment was to check if it is possible to ensure real-time tracking with at least 25 processed frames per second for standard definition video. OpenCV version 4.5.0 was used with 7 tracking algorithms from the extra modules package. They are: Boosting, CSRT, KCF, MedianFlow, MIL, MOSSE, TLD. More than 5600 frames of standard definition were processed during the experiment. Analysis of the results shows that two algorithms-CSRT and KCF-could solve the problem of tumor tracking. They lead the test with 70% and more of Intersection over Union and more than 85% successful searches. They could also be used in real-time processing with an average processing speed of up to frames per second in CSRT and 100 + frames per second for KCF. Tracking results reach the average deviation between centers of neoplasms to 2 mm and maximum deviation less than 5 mm. This experiment also shows that no frames made CSRT and KCF algorithms fail simultaneously. So, the hypothesis for future work is combining these algorithms to work together, with one of them-CSRT-as support for the KCF tracker on the rarely failed frames.

Experimental Validation of a Real-Time Adaptive 4D-Optimized Particle Radiotherapy Approach to Treat Irregularly Moving Tumors

PURPOSE

Treatment of locally advanced lung cancer is limited by toxicity and insufficient local control. Particle therapy could enable more conformal treatment than intensity modulated photon therapy but is challenged by irregular tumor motion, associated range changes, and tumor deformations. We propose a new strategy for robust, online adaptive particle therapy, synergizing 4-dimensional optimization with real-time adaptive beam tracking. The strategy was tested and the required motion monitoring precision was determined.

METHODS AND MATERIALS

In multiphase 4-dimensional dose delivery (MP4D), a dedicated quasistatic treatment plan is delivered to each motion phase of periodic 4-dimensional computed tomography (4DCT). In the new extension, "MP4D with residual tracking" (MP4DRT), lateral beam tracking compensates for the displacement of the tumor center-of-mass relative to the current phase in the planning 4DCT. We implemented this method in the dose delivery system of a clinical carbon facility and tested it experimentally for a lung cancer plan based on a periodic subset of a virtual lung 4DCT (planned motion amplitude 20 mm). Treatments were delivered in a quality assurance-like setting to a moving ionization chamber array. We considered variable motion amplitudes and baseline drifts. The required motion monitoring precision was evaluated by adding noise to the motion signal. Log-file-based dose reconstructions were performed in silico on the entire 4DCT phantom data set capable of simulating nonperiodic motion. MP4DRT was compared with MP4D, rescanned beam tracking, and internal target volume plans. Treatment quality was assessed in terms of target coverage (D95), dose homogeneity (D5-D95), conformity number, and dose to heart and lung.

RESULTS

For all considered motion scenarios and metrics, MP4DRT produced the most favorable metrics among the tested motion mitigation strategies and delivered high-quality treatments. The conformity was similar to static treatments. The motion monitoring precision required for D95 >95% was 1.9 mm.

CONCLUSIONS

With clinically feasible motion monitoring, MP4DRT can deliver highly conformal dose distributions to irregularly moving targets.

Multi3: multi-templates siamese network with multi-peaks detection and multi-features refinement for target tracking in ultrasound image sequences

Objective. Radiation therapy requires a precise target location. However, respiratory motion increases the uncertainties of the target location. Accurate and robust tracking is significant for improving operation accuracy. Approach. In this work, we propose a tracking framework Multi3, including a multi-templates Siamese network, multi-peaks detection and multi-features refinement, for target tracking in ultrasound sequences. Specifically, we use two templates to provide the location and deformation of the target for robust tracking. Multi-peaks detection is applied to extend the set of potential target locations, and multi-features refinement is designed to select an appropriate location as the tracking result by quality assessment. Main results. The proposed Multi3 is evaluated on a public dataset, i.e. MICCAI 2015 challenge on liver ultrasound tracking (CLUST), and our clinical dataset provided by the Chinese People’s Liberation Army General Hospital. Experimental results show that Multi3 achieves accurate and robust tracking in ultrasound sequences (0.75 ± 0.62 mm and 0.51 ± 0.32 mm tracking errors in two datasets). Significance. The proposed Multi3 is the most robust method on the CLUST 2D benchmark set, exhibiting potential in clinical practice.

A phantom-based analysis for tracking intra-fraction pancreatic tumor motion by ultrasound imaging during radiation therapy

Purpose

In this study, we aim to further evaluate the accuracy of ultrasound tracking for intra-fraction pancreatic tumor motion during radiotherapy by a phantom-based study.

Methods

Twelve patients with pancreatic cancer who were treated with stereotactic body radiation therapy were enrolled in this study. The displacement points of the respiratory cycle were acquired from 4DCT and transferred to a motion platform to mimic realistic breathing movements in our phantom study. An ultrasound abdominal phantom was placed and fixed in the motion platform. The ground truth of phantom movement was recorded by tracking an optical tracker attached to this phantom. One tumor inside the phantom was the tracking target. In the evaluation of the results, the monitoring results from the ultrasound system were compared with the phantom motion results from the infrared camera. Differences between infrared monitoring motion and ultrasound tracking motion were analyzed by calculating the root-mean-square error.

Results

The 82.2% ultrasound tracking motion was within a 0.5 mm difference value between ultrasound tracking displacement and infrared monitoring motion. 0.7% ultrasound tracking failed to track accurately (a difference value > 2.5 mm). These differences between ultrasound tracking motion and infrared monitored motion do not correlate with respiratory displacements, respiratory velocity, or respiratory acceleration by linear regression analysis.

Conclusions

The highly accurate monitoring results of this phantom study prove that the ultrasound tracking system may be a potential method for real-time monitoring targets, allowing more accurate delivery of radiation doses.

Therapeutic Ultrasound Effects on Human Induced Pluripotent Stem Cell Cardiomyocytes Measured Optically and with Spectral Ultrasound

To the best of our knowledge, therapeutic ultrasound (TUS) is thus far an unexplored means of delivering mechanical stimulation to cardiomyocyte cultures, which is necessary to engineer a more mature cardiomyocyte phenotype in vitro. Spectral ultrasound (SUS) may provide a way to non-invasively, non-disruptively and inexpensively monitor growth and change in cell cultures over long periods. Compared with other measurement methods, SUS as an acoustic measurement tool will not be affected by an acoustic therapy, unlike electrical measurement methods, in which motion caused by acoustic therapy can affect measurements. Further SUS has the potential to provide functional as well as morphological information in cell cultures. Human induced pluripotent stem cell cardiomyocytes (iPS-CMs) were imaged with calcium fluorescence microscopy while TUS was being applied. TUS was applied at 600 kHz and 1, 3.4 and 6 W/cm² for a continuous 1 s pulse. Measures of the instantaneous beat frequency, repolarization rate and calcium spike amplitude were calculated from the fluorescence data. At 600 kHz, TUS at 1 and 6 W/cm² had significant effects on the shortening of both the repolarization rate and instantaneous beat rate of the iPS-CMs (p < 0.05), while TUS at 3.4 and 6 W/cm² had significant effects on the shortening of the calcium spike amplitude (p < 0.05). Three SUS measures and one gray-level measure were captured from the iPS-CM monolayers while they were simultaneously being imaged with calcium-labeled confocal microscopy. The gray-level measure performed the best of all SUS measures; however, it was not reliable enough to produce a consistent determination of the beat rate of the cell. Finally, SUS measures were captured using three different transducers while simultaneously applying TUS. A center-of-mass (COM) measure calculated from the wavelet transform scalogram of the time-averaged radiofrequency data revealed that SUS was able to detect a change in the frequency content of the reflected ultrasound at 1 and 6 W/cm² before and after ultrasound application (p < 0.05), showing promise for the ability of SUS to measure changes in the beating behavior of iPS-CMs. Overall, SUS is promising as a method for constant monitoring of dynamic cell and tissue culture and growth.

Demonstrating the benefits of corrective intraoperative feedback in improving the quality of duodenal hydrogel spacer placement

Purpose

Pancreatic cancer is the fourth leading cause of cancer‐related death with a 10% 5‐year overall survival rate (OS). Radiation therapy (RT) in addition to dose escalation improves the outcome by significantly increasing the OS at 2 and 3 years but is hindered by the toxicity of the duodenum. Our group showed that the insertion of hydrogel spacer reduces duodenal toxicity, but the complex anatomy and the demanding procedure make the benefits highly uncertain. Here, we investigated the feasibility of augmenting the workflow with intraoperative feedback to reduce the adverse effects of the uncertainties.

Materials and Methods

We simulated three scenarios of the virtual spacer for four cadavers with two types of gross tumor volume (GTV) (small and large); first, the ideal injection; second, the nonideal injection that incorporates common spacer placement uncertainties; and third, the corrective injection that uses the simulation result from nonideal injection and is designed to compensate for the effect of uncertainties. We considered two common uncertainties: (1) “Narrowing” is defined as the injection of smaller spacer volume than planned. (2) “Missing part” is defined as failure to inject spacer in the ascending section of the duodenum. A total of 32 stereotactic body radiation therapy (SBRT) plans (33 Gy in 5 fractions) were designed, for four cadavers, two GTV sizes, and two types of uncertainties. The preinjection scenario for each case was compared with three scenarios of virtual spacer placement from the dosimetric and geometric points of view.

Results

We found that the overlapping PTV space with the duodenum is an informative quantity for determining the effective location of the spacer. The ideal spacer distribution reduced the duodenal V33Gy for small and large GTV to less than 0.3 and 0.1cc, from an average of 3.3cc, and 1.2cc for the preinjection scenario. However, spacer placement uncertainties reduced the efficacy of the spacer in sparing the duodenum (duodenal V33Gy: 1.3 and 0.4cc). The separation between duodenum and GTV decreased by an average of 5.3 and 4.6 mm. The corrective feedback can effectively bring back the expected benefits from the ideal location of the spacer (averaged V33Gy of 0.4 and 0.1cc).

Conclusions

An informative feedback metric was introduced and used to mitigate the effect of spacer placement uncertainties and maximize the benefits of the EUS‐guided procedure.

Fusion Siamese network with drift correction for target tracking in ultrasound sequences

Motion tracking techniques can revise the bias arising from respiration-caused motion in radiation therapy. Tracking key structures accurately and at a real-time speed is necessary for effective motion tracking. In this work, we propose a fusion Siamese network with drift correction for target tracking in ultrasound sequences. Specifically, the network fuses four response maps generated by the cross-correlation between convolution layers at different resolutions to reduce up-sampling error. A correction strategy combining local structural similarity and target trajectory is proposed to revise the target drift predicted by the network. Moreover, a coarse-to-fine strategy is proposed to train the network with a limited number of annotated images, in which an augmented dataset is generated by corner points to learn network features with high generalizability. The proposed method is evaluated on the basis of the public dataset of the MICCAI 2015 Challenge on Liver UltraSound Tracking (CLUST) and our ultrasound image dataset, which is provided by the Chinese People’s Liberation Army General Hospital (CPLAGH). A tracking error of 0.80 ± 1.16 mm is observed for 85 targets across 39 ultrasound sequences in the CLUST dataset. A tracking error of 0.61 ± 0.36 mm is observed for 20 targets across 10 ultrasound sequences in the CPLAGH dataset. The effectiveness of the proposed fusion and correction strategies is verified via two ablation experiments. Overall, the experimental results demonstrate the effectiveness of the proposed fusion Siamese network with drift correction and reveal its potential in clinical practice.

Deep learning-based motion tracking using ultrasound images

PURPOSE

Ultrasound (US) imaging is an established imaging modality capable of offering video-rate volumetric images without ionizing radiation. It has the potential for intra-fraction motion tracking in radiation therapy. In this study, a deep learning-based method has been developed to tackle the challenges in motion tracking using US imaging.

METHODS

We present a Markov-like network, which is implemented via generative adversarial networks, to extract features from sequential US frames (one tracked frame followed by untracked frames) and thereby estimate a set of deformation vector fields (DVFs) through the registration of the tracked frame and the untracked frames. The positions of the landmarks in the untracked frames are finally determined by shifting landmarks in the tracked frame according to the estimated DVFs. The performance of the proposed method was evaluated on the testing dataset by calculating the tracking error (TE) between the predicted and ground truth landmarks on each frame.

RESULTS

The proposed method was evaluated using the MICCAI CLUST 2015 dataset which was collected using seven US scanners with eight types of transducers and the Cardiac Acquisitions for Multi-structure Ultrasound Segmentation (CAMUS) dataset which was acquired using GE Vivid E95 ultrasound scanners. The CLUST dataset contains 63 2D and 22 3D US image sequences respectively from 42 and 18 subjects, and the CAMUS dataset includes 2D US images from 450 patients. On CLUST dataset, our proposed method achieved a mean tracking error of 0.70 ± 0.38 mm for the 2D sequences and 1.71 ± 0.84 mm for the 3D sequences for those public available annotations. And on CAMUS dataset, a mean tracking error of 0.54 ± 1.24 mm for the landmarks in the left atrium was achieved.

CONCLUSIONS

A novel motion tracking algorithm using US images based on modern deep learning techniques has been demonstrated in this study. The proposed method can offer millimeter-level tumor motion prediction in real time, which has the potential to be adopted into routine tumor motion management in radiation therapy.

Deep Learning for Ultrasound Beamforming in Flexible Array Transducer

Ultrasound imaging has been developed for image-guided radiotherapy for tumor tracking, and the flexible array transducer is a promising tool for this task. It can reduce the user dependence and anatomical changes caused by the traditional ultrasound transducer. However, due to its flexible geometry, the conventional delay-and-sum (DAS) beamformer may apply incorrect time delay to the radio-frequency (RF) data and produce B-mode images with considerable defocusing and distortion. To address this problem, we propose a novel end-to-end deep learning approach that may alternate the conventional DAS beamformer when the transducer geometry is unknown. Different deep neural networks (DNNs) were designed to learn the proper time delays for each channel, and they were expected to reconstruct the undistorted high-quality B-mode images directly from RF channel data. We compared the DNN results to the standard DAS beamformed results using simulation and flexible array transducer scan data. With the proposed DNN approach, the averaged full-width-at-half-maximum (FWHM) of point scatters is 1.80 mm and 1.31 mm lower in simulation and scan results, respectively; the contrast-to-noise ratio (CNR) of the anechoic cyst in simulation and phantom scan is improved by 0.79 dB and 1.69 dB, respectively; and the aspect ratios of all the cysts are closer to 1. The evaluation results show that the proposed approach can effectively reduce the distortion and improve the lateral resolution and contrast of the reconstructed B-mode images.

Deep Learning to Obtain Simultaneous Image and Segmentation Outputs From a Single Input of Raw Ultrasound Channel Data

Single plane wave transmissions are promising for automated imaging tasks requiring high ultrasound frame rates over an extended field of view. However, a single plane wave insonification typically produces suboptimal image quality. To address this limitation, we are exploring the use of deep neural networks (DNNs) as an alternative to delay-and-sum (DAS) beamforming. The objectives of this work are to obtain information directly from raw channel data and to simultaneously generate both a segmentation map for automated ultrasound tasks and a corresponding ultrasound B-mode image for interpretable supervision of the automation. We focus on visualizing and segmenting anechoic targets surrounded by tissue and ignoring or deemphasizing less important surrounding structures. DNNs trained with Field II simulations were tested with simulated, experimental phantom, and in vivo data sets that were not included during training. With unfocused input channel data (i.e., prior to the application of receive time delays), simulated, experimental phantom, and in vivo test data sets achieved mean ± standard deviation Dice similarity coefficients of 0.92 ± 0.13, 0.92 ± 0.03, and 0.77 ± 0.07, respectively, and generalized contrast-to-noise ratios (gCNRs) of 0.95 ± 0.08, 0.93 ± 0.08, and 0.75 ± 0.14, respectively. With subaperture beamformed channel data and a modification to the input layer of the DNN architecture to accept these data, the fidelity of image reconstruction increased (e.g., mean gCNR of multiple acquisitions of two in vivo breast cysts ranged 0.89-0.96), but DNN display frame rates were reduced from 395 to 287 Hz. Overall, the DNNs successfully translated feature representations learned from simulated data to phantom and in vivo data, which is promising for this novel approach to simultaneous ultrasound image formation and segmentation.

Cascaded one-shot deformable convolutional neural networks: Developing a deep learning model for respiratory motion estimation in ultrasound sequences

Improving the quality of image-guided radiation therapy requires the tracking of respiratory motion in ultrasound sequences. However, the low signal-to-noise ratio and the artifacts in ultrasound images make it difficult to track targets accurately and robustly. In this study, we propose a novel deep learning model, called a Cascaded One-shot Deformable Convolutional Neural Network (COSD-CNN), to track landmarks in real time in long ultrasound sequences. Specifically, we design a cascaded Siamese network structure to improve the tracking performance of CNN-based methods. We propose a one-shot deformable convolution module to enhance the robustness of the COSD-CNN to appearance variation in a meta-learning manner. Moreover, we design a simple and efficient unsupervised strategy to facilitate the network's training with a limited number of medical images, in which many corner points are selected from raw ultrasound images to learn network features with high generalizability. The proposed COSD-CNN has been extensively evaluated on the public Challenge on Liver UltraSound Tracking (CLUST) 2D dataset and on our own ultrasound image dataset from the First Affiliated Hospital of Sun Yat-sen University (FSYSU). Experiment results show that the proposed model can track a target through an ultrasound sequence with high accuracy and robustness. Our method achieves new state-of-the-art performance on the CLUST 2D benchmark set, indicating its strong potential for application in clinical practice.