Robotic intrafractional US guidance for liver SABR: System design, beam avoidance, and clinical imaging

Increasing Reachability in Robotic Ultrasound Through Base Placement and Tool Design

BACKGROUND

Robotic ultrasound visualises internal organs in real-time for various medical applications without the harm of X-rays. The ultrasound probe is attached to the robot's end effector using custom-developed probe holders. This paper analyzes the impact of different probe holder geometries on the robot's base placement and reachability.

METHODS

We propose a method to improve probe holder geometries and robot base placements to enhance reachability, validated using a 7-DoF serial manipulator (KUKA iiwa 7) for ultrasound scans of multiple subcutaneous body parts.

RESULTS

Without additional space restrictions, the number of robot base positions with high reachability could be strongly increased with an improved probe holder geometry. Under space constraints, previously unreachable target poses became accessible by adapting the probe holder geometry.

CONCLUSIONS

Our method provides an automated solution for determining improved probe holder geometries, enhancing reachability to target areas, especially when the robot's placing area is limited.

Current Research Status of Respiratory Motion for Thorax and Abdominal Treatment: A Systematic Review

Malignant tumors have become one of the serious public health problems in human safety and health, among which the chest and abdomen diseases account for the largest proportion. Early diagnosis and treatment can effectively improve the survival rate of patients. However, respiratory motion in the chest and abdomen can lead to uncertainty in the shape, volume, and location of the tumor, making treatment of the chest and abdomen difficult. Therefore, compensation for respiratory motion is very important in clinical treatment. The purpose of this review was to discuss the research and development of respiratory movement monitoring and prediction in thoracic and abdominal surgery, as well as introduce the current research status. The integration of modern respiratory motion compensation technology with advanced sensor detection technology, medical-image-guided therapy, and artificial intelligence technology is discussed and analyzed. The future research direction of intraoperative thoracic and abdominal respiratory motion compensation should be non-invasive, non-contact, use a low dose, and involve intelligent development. The complexity of the surgical environment, the constraints on the accuracy of existing image guidance devices, and the latency of data transmission are all present technical challenges.

Real-Time Motion Analysis With 4D Deep Learning for Ultrasound-Guided Radiotherapy

Motion compensation in radiation therapy is a challenging scenario that requires estimating and forecasting motion of tissue structures to deliver the target dose. Ultrasound offers direct imaging of tissue in real-time and is considered for image guidance in radiation therapy. Recently, fast volumetric ultrasound has gained traction, but motion analysis with such high-dimensional data remains difficult. While deep learning could bring many advantages, such as fast data processing and high performance, it remains unclear how to process sequences of hundreds of image volumes efficiently and effectively. We present a 4D deep learning approach for real-time motion estimation and forecasting using long-term 4D ultrasound data. Using motion traces acquired during radiation therapy combined with various tissue types, our results demonstrate that long-term motion estimation can be performed markerless with a tracking error of 0.35±0.2 mm and with an inference time of less than 5 ms. Also, we demonstrate forecasting directly from the image data up to 900 ms into the future. Overall, our findings highlight that 4D deep learning is a promising approach for motion analysis during radiotherapy.

AI-based optimization for US-guided radiation therapy of the prostate

OBJECTIVES

Fast volumetric ultrasound presents an interesting modality for continuous and real-time intra-fractional target tracking in radiation therapy of lesions in the abdomen. However, the placement of the ultrasound probe close to the target structures leads to blocking some beam directions.

METHODS

To handle the combinatorial complexity of searching for the ultrasound-robot pose and the subset of optimal treatment beams, we combine CNN-based candidate beam selection with simulated annealing for setup optimization of the ultrasound robot, and linear optimization for treatment plan optimization into an AI-based approach. For 50 prostate cases previously treated with the CyberKnife, we study setup and treatment plan optimization when including robotic ultrasound guidance.

RESULTS

The CNN-based search substantially outperforms previous randomized heuristics, increasing coverage from 93.66 to 97.20% on average. Moreover, in some cases the total MU was also reduced, particularly for smaller target volumes. Results after AI-based optimization are similar for treatment plans with and without beam blocking due to ultrasound guidance.

CONCLUSIONS

AI-based optimization allows for fast and effective search for configurations for robotic ultrasound-guided radiation therapy. The negative impact of the ultrasound robot on the plan quality can successfully be mitigated resulting only in minor differences.

Case Report: Treatment Planning Study to Demonstrate Feasibility of Transthoracic Ultrasound Guidance to Facilitate Ventricular Tachycardia Ablation With Protons

Background

Cardiac arrhythmias, such as ventricular tachycardia, are disruptions in the normal cardiac function that originate from problems in the electrical conduction of signals inside the heart. Recently, a non-invasive treatment option based on external photon or proton beam irradiation has been used to ablate the arrhythmogenic structures. Especially in proton therapy, based on its steep dose gradient, it is crucial to monitor the motion of the heart in order to ensure that the radiation dose is delivered to the correct location. Transthoracic ultrasound imaging has the potential to provide guidance during this treatment delivery. However, it has to be noted that the presence of an ultrasound probe on the chest of the patient introduces constraints on usable beam angles for both protons and photon treatments. This case report investigates the possibility to generate a clinically acceptable proton treatment plan while the ultrasound probe is present on the chest of the patient.

Case

A treatment plan study was performed based on a 4D cardiac-gated computed tomography scan of a 55 year-old male patient suffering from refractory ventricular tachycardia who underwent cardiac radioablation. A proton therapy treatment plan was generated for the actual treatment target in presence of an ultrasound probe on the chest of this patient. The clinical acceptability of the generated plan was confirmed by evaluating standard target dose-volume metrics, dose to organs-at-risk and target dose conformity and homogeneity.

Conclusion

The generation of a clinically acceptable proton therapy treatment plan for cardiac radioablation of ventricular tachycardia could be performed in the presence of an ultrasound probe on the chest of the patient. These results establish a basis and justification for continued research and product development for ultrasound-guided cardiac radioablation.

Interpretable Machine Learning for Characterization of Focal Liver Lesions by Contrast-Enhanced Ultrasound

This work proposes an interpretable radiomics approach to differentiate between malignant and benign focal liver lesions (FLLs) on contrast-enhanced ultrasound (CEUS). Although CEUS has shown promise for differential FLLs diagnosis, current clinical assessment is performed only by qualitative analysis of the contrast enhancement patterns. Quantitative analysis is often hampered by the unavoidable presence of motion artifacts and by the complex, spatiotemporal nature of liver contrast enhancement, consisting of multiple, overlapping vascular phases. To fully exploit the wealth of information in CEUS, while coping with these challenges, here we propose combining features extracted by the temporal and spatiotemporal analysis in the arterial phase enhancement with spatial features extracted by texture analysis at different time points. Using the extracted features as input, several machine learning classifiers are optimized to achieve semiautomatic FLLs characterization, for which there is no need for motion compensation and the only manual input required is the location of a suspicious lesion. Clinical validation on 87 FLLs from 72 patients at risk for hepatocellular carcinoma (HCC) showed promising performance, achieving a balanced accuracy of 0.84 in the distinction between benign and malignant lesions. Analysis of feature relevance demonstrates that a combination of spatiotemporal and texture features is needed to achieve the best performance. Interpretation of the most relevant features suggests that aspects related to microvascular perfusion and the microvascular architecture, together with the spatial enhancement characteristics at wash-in and peak enhancement, are important to aid the accurate characterization of FLLs.

Towards automated ultrasound imaging-robotic image acquisition in liver and prostate for long-term motion monitoring

Real-time volumetric (4D) ultrasound has shown high potential for diagnostic and therapy guidance tasks. One of the main drawbacks of ultrasound imaging to date is the reliance on manual probe positioning and the resulting user dependence. Robotic assistance could help overcome this issue and facilitate the acquisition of long-term image data to observe dynamic processesin vivoover time. The aim of this study is to assess the feasibility of robotic probe manipulation and organ motion quantification during extended imaging sessions. The system consists of a collaborative robot and a 4D ultrasound system providing real-time data access. Five healthy volunteers received liver and prostate scans during free breathing over 30 min. Initial probe placement was performed with real-time remote control with a predefined contact force of 10 N. During scan acquisition, the probe position was continuously adjusted to the body surface motion using impedance control. Ultrasound volumes, the pose of the end-effector and the estimated contact forces were recorded. For motion analysis, one anatomical landmark was manually annotated in a subset of ultrasound frames for each experiment. Probe contact was uninterrupted over the entire scan duration in all ten sessions. Organ drift and imaging artefacts were successfully compensated using remote control. The median contact force along the probe's longitudinal axis was 10.0 N with maximum values of 13.2 and 21.3 N for liver and prostate, respectively. Forces exceeding 11 N only occurred in 0.3% of the time. Probe and landmark motion were more pronounced in the liver, with median interquartile ranges of 1.5 and 9.6 mm, compared to 0.6 and 2.7 mm in the prostate. The results show that robotic ultrasound imaging with dynamic force control can be used for stable, long-term imaging of anatomical regions affected by motion. The system facilitates the acquisition of 4D image datain vivoover extended scanning periods for the first time and holds the potential to be used for motion monitoring for therapy guidance as well as diagnostic tasks.

Medical Robotics for Ultrasound Imaging: Current Systems and Future Trends

Purpose of Review

This review provides an overview of the most recent robotic ultrasound systems that have contemporary emerged over the past five years, highlighting their status and future directions. The systems are categorized based on their level of robot autonomy (LORA).

Recent Findings

Teleoperating systems show the highest level of technical maturity. Collaborative assisting and autonomous systems are still in the research phase, with a focus on ultrasound image processing and force adaptation strategies. However, missing key factors are clinical studies and appropriate safety strategies. Future research will likely focus on artificial intelligence and virtual/augmented reality to improve image understanding and ergonomics.

Summary

A review on robotic ultrasound systems is presented in which first technical specifications are outlined. Hereafter, the literature of the past five years is subdivided into teleoperation, collaborative assistance, or autonomous systems based on LORA. Finally, future trends for robotic ultrasound systems are reviewed with a focus on artificial intelligence and virtual/augmented reality.

Target tracking accuracy and latency with different 4D ultrasound systems – a robotic phantom study

Ultrasound (US) imaging, in contrast to other image guidance techniques, offers the distinct advantage of providing volumetric image data in real-time (4D) without using ionizing radiation. The goal of this study was to perform the first quantitative comparison of three different 4D US systems with fast matrix array probes and real-time data streaming regarding their target tracking accuracy and system latency. Sinusoidal motion of varying amplitudes and frequencies was used to simulate breathing motion with a robotic arm and a static US phantom. US volumes and robot positions were acquired online and stored for retrospective analysis. A template matching approach was used for target localization in the US data. Target motion measured in US was compared to the reference trajectory performed by the robot to determine localization accuracy and system latency. Using the robotic setup, all investigated 4D US systems could detect a moving target with sub-millimeter accuracy. However, especially high system latency increased tracking errors substantially and should be compensated with prediction algorithms for respiratory motion compensation.

Optimizing Configurations for 7-DoF Robotic Ultrasound Guidance in Radiotherapy of the Prostate

Robotic ultrasound guidance is promising for tracking of organ motion during radiotherapy treatments, but the radio-opaque robot and probe interfere with beam delivery. The effect on treatment plan quality can be mitigated by the use of a robot arm with kinematic redundancy, such that the robot is able to elude delivered beams during treatment by changing its configuration. However, these changes require robot motion close to the patient, lead to an increased treatment time, and require coordination with the beam delivery. We propose an optimization workflow which integrates the problem of selecting suitable robot configurations into a linear-programming-based workflow for treatment plan optimization. Starting with a large set of candidate configurations, a minimal subset is determined which provides equivalent plan quality. Our results show that, typically, six configurations are sufficient for this purpose. Furthermore, we show that optimal configurations can be reused for dose planning of subsequent patients.

2D ultrasound imaging based intra-fraction respiratory motion tracking for abdominal radiation therapy using machine learning

We have previously developed a robotic ultrasound imaging system for motion monitoring in abdominal radiation therapy. Owing to the slow speed of ultrasound image processing, our previous system could only track abdominal motions under breath-hold. To overcome this limitation, a novel 2D-based image processing method for tracking intra-fraction respiratory motion is proposed. Fifty-seven different anatomical features acquired from 27 sets of 2D ultrasound sequences were used in this study. Three 2D ultrasound sequences were acquired with the robotic ultrasound system from three healthy volunteers. The remaining datasets were provided by the 2015 MICCAI Challenge on Liver Ultrasound Tracking. All datasets were preprocessed to extract the feature point, and a patient-specific motion pattern was extracted by principal component analysis and slow feature analysis (SFA). The tracking finds the most similar frame (or indexed frame) by a k-dimensional-tree-based nearest neighbor search for estimating the tracked object location. A template image was updated dynamically through the indexed frame to perform a fast template matching (TM) within a learned smaller search region on the incoming frame. The mean tracking error between manually annotated landmarks and the location extracted from the indexed training frame is 1.80  ±  1.42 mm. Adding a fast TM procedure within a small search region reduces the mean tracking error to 1.14  ±  1.16 mm. The tracking time per frame is 15 ms, which is well below the frame acquisition time. Furthermore, the anatomical reproducibility was measured by analyzing the location's anatomical landmark relative to the probe; the position-controlled probe has better reproducibility and yields a smaller mean error across all three volunteer cases, compared to the force-controlled probe (2.69 versus 11.20 mm in the superior-inferior direction and 1.19 versus 8.21 mm in the anterior-posterior direction). Our method reduces the processing time for tracking respiratory motion significantly, which can reduce the delivery uncertainty.

Real-time intrafraction motion monitoring in external beam radiotherapy

Radiotherapy (RT) aims to deliver a spatially conformal dose of radiation to tumours while maximizing the dose sparing to healthy tissues. However, the internal patient anatomy is constantly moving due to respiratory, cardiac, gastrointestinal and urinary activity. The long term goal of the RT community to 'see what we treat, as we treat' and to act on this information instantaneously has resulted in rapid technological innovation. Specialized treatment machines, such as robotic or gimbal-steered linear accelerators (linac) with in-room imaging suites, have been developed specifically for real-time treatment adaptation. Additional equipment, such as stereoscopic kilovoltage (kV) imaging, ultrasound transducers and electromagnetic transponders, has been developed for intrafraction motion monitoring on conventional linacs. Magnetic resonance imaging (MRI) has been integrated with cobalt treatment units and more recently with linacs. In addition to hardware innovation, software development has played a substantial role in the development of motion monitoring methods based on respiratory motion surrogates and planar kV or Megavoltage (MV) imaging that is available on standard equipped linacs. In this paper, we review and compare the different intrafraction motion monitoring methods proposed in the literature and demonstrated in real-time on clinical data as well as their possible future developments. We then discuss general considerations on validation and quality assurance for clinical implementation. Besides photon RT, particle therapy is increasingly used to treat moving targets. However, transferring motion monitoring technologies from linacs to particle beam lines presents substantial challenges. Lessons learned from the implementation of real-time intrafraction monitoring for photon RT will be used as a basis to discuss the implementation of these methods for particle RT.

A visual probe positioning tool for 4D ultrasound-guided radiotherapy

Ultrasound (US) guidance is a rapidly growing area in image-guided radiotherapy. For motion compensation, the therapy target needs to be visualized with the US probe to continuously determine its position and adapt for shifts. While US has obvious benefits such as real-time capability and proven safety, one of the main drawbacks to date is its user dependency - high quality results require long years of clinical experience. To provide positioning assistance for the setup of US equipment by non-experts, we developed a visual guidance tool combining real-time US volume and CT visualization in a geometrically calibrated setup. By using a 4D US station with real-time data access and an optical tracking system, we achieved a calibration accuracy of 1.2 mm and a mean 2D contour distance of 1.7 mm between organ boundaries identified in US and CT. With this low calibration error as well as the good visual alignment of the structures, the developed probe positioning tool could be a valuable aid for ultrasound-guided radiotherapy and other interventions by guiding the user to a suitable acoustic window while potentially improving setup reproducibility.