Robot-assisted automatic ultrasound calibration

Strategy for automatic ultrasound (US) probe positioning in robot-assisted ultrasound guided radiation therapy

Objective. As part of image-guided radiotherapy, ultrasound-guided radiotherapy is currently already in use and under investigation for robot assisted systems Ipsen 2021. It promises a real-time tumor localization during irradiation (intrafractional) without extra dose. The ultrasound probe is held and guided by a robot. However, there is a lack of basic safety mechanisms and interaction strategies to enable a safe clinical procedure. In this study we investigate potential positioning strategies with safety mechanisms for a safe robot-human-interaction.Approach. A compact setup of ultrasound device, lightweight robot, tracking camera, force sensor and control computer were integrated in a software application to represent a potential USgRT setup. For the realization of a clinical procedure, positioning strategies for the ultrasound head with the help of the robot were developed, implemented, and tested. In addition, basic safety mechanisms for the robot have been implemented, using the integrated force sensor, and have been tested by intentional collisions.Main results. Various positioning methods from manual guidance to completely automated procedures were tested. Robot-guided methods achieved higher positioning accuracy and were faster in execution compared to conventional hand-guided methods. The developed safety mechanisms worked as intended and the detected collision force were below 20 N.Significance. The study demonstrates the feasibility of a new approach for safe robotic ultrasound imaging, with a focus on abdominal usage (liver, prostate, kidney). The safety measures applied here can be extended to other human-robot interactions and present the basic for further studies in medical applications.

Calibration method for a breast intervention robot based on four-dimensional ultrasound image guidance

In breast interventional ultrasound therapy, it is difficult to directly diagnose the location of a tumor in 2-D ultrasound images. To assist surgeons in treatment more intuitively, a four-dimensional ultrasound image-guided breast intervention robot is proposed. The calibration approach of the ultrasonic image for the robot is one of the main contents of the research. This method is based on the establishment of a complete coordinate system conversion model, and it uses the ORB (oriented FAST and rotated BRIEF) feature extraction method to obtain and record the real-time image marker pixel positions, calculate the unknown parameters of the coordinate system conversion matrix, and establish a complete calibration system. This article demonstrates the feasibility of the calibration approach through experiments in our developed US-guided robotic system. Additional experimental and parametrical comparisons of the proposed method with state-of-the-art methods were conducted to thoroughly evaluate the outperformance of the proposed method.

Ultrasound Calibration for Dual-Armed Surgical Navigation System

Ultrasound (US) imaging system is widely used in robotic systems for precision positioning in clinical applications. The US calibration is critical to minimize the difference of spatial coordinates between instruments, for minimally invasive surgery (MIS) in navigation systems. In this study, we propose a dual robotic arm system that combines US imaging with one arm for path planning and monitoring and accurate positioning with the other arm for instrument placement via the preplanning procedures. A phantom with N-wire and N-wedge was designed for US calibration. The US calibration showed a mean error of 0.76 mm; the mean dual-arm calibration error is 0.31 mm. The positioning error of the system was verified with a mean error of 1.48 mm. In addition, we used two abdominal phantoms with computed tomography scan validation, with an averaged position error of 1.867 ± 0.436 mm and an orientation error of 2.190 ± 0.764°. The proposed system is aimed to perform clinical operations, such as abdominal MIS, with real-time image monitoring of the organ tissues and instrument positions, which meet the requirements for medical application.

Automatic Frame Selection using CNN in Ultrasound Elastography

Ultrasound elastography is used to estimate the mechanical properties of the tissue by monitoring its response to an internal or external force. Different levels of deformation are obtained from different tissue types depending on their mechanical properties, where stiffer tissues deform less. Given two radio frequency (RF) frames collected before and after some deformation, we estimate displacement and strain images by comparing the RF frames. The quality of the strain image is dependent on the type of motion that occurs during deformation. In-plane axial motion results in high-quality strain images, whereas out-of-plane motion results in low-quality strain images. In this paper, we introduce a new method using a convolutional neural network (CNN) to determine the suitability of a pair of RF frames for elastography in only 5.4 ms. Our method could also be used to automatically choose the best pair of RF frames, yielding a high-quality strain image. The CNN was trained on 3,818 pairs of RF frames, while testing was done on 986 new unseen pairs, achieving an accuracy of more than 91%. The RF frames were collected from both phantom and in vivo data.

Phantom with multiple active points for ultrasound calibration

Accurate tracking and localization of ultrasound (US) images are used in various computer-assisted interventions. US calibration is a preoperative procedure to recover the transformation bridging the tracking sensor and the US image coordinate systems. Although many calibration phantom designs have been proposed, a limitation that hinders the resulted calibration accuracy is US elevational beam thickness. Previous studies have proposed an active-echo (AE)-based calibration concept to overcome this limitation by utilizing dynamic active US feedback from a single PZT element-based phantom, which assists in placing the phantom within the US elevational plane. However, the process of searching elevational midplane is time-consuming and requires dedicated hardware to enable "AE" functionality. Extending this active phantom, we present a US calibration concept and associated mathematical framework enabling fast and accurate US calibration using multiple "active" points. The proposed US calibration can simplify the calibration procedure by minimizing the number of times midplane search is performed and shortening calibration time. This concept is demonstrated with a configuration mechanically tracking a US probe using a robot arm. We validated the concept through simulation and experiment, and achieved submillimeter calibration accuracy. This result indicates that the multiple active-point phantom has potential to provide superior calibration performance for applications requiring high tracking accuracy.

Active phantoms: a paradigm for ultrasound calibration using phantom feedback

In ultrasound (US)-guided medical procedures, accurate tracking of interventional tools is crucial to patient safety and clinical outcome. This requires a calibration procedure to recover the relationship between the US image and the tracking coordinate system. In literature, calibration has been performed on passive phantoms, which depend on image quality and parameters, such as frequency, depth, and beam-thickness as well as in-plane assumptions. In this work, we introduce an active phantom for US calibration. This phantom actively detects and responds to the US beams transmitted from the imaging probe. This active echo (AE) approach allows identification of the US image midplane independent of image quality. Both target localization and segmentation can be done automatically, minimizing user dependency. The AE phantom is compared with a crosswire phantom in a robotic US setup. An out-of-plane estimation US calibration method is also demonstrated through simulation and experiments to compensate for remaining elevational uncertainty. The results indicate that the AE calibration phantom can have more consistent results across experiments with varying image configurations. Automatic segmentation is also shown to have similar performance to manual segmentation.

Surgical fiducial segmentation and tracking for pose estimation based on ultrasound B-mode images

Doppler ultrasound is a non-invasive diagnostic tool for the quantitative measurement of blood flow. However, given that it provides velocity data that is dependent on the location and angle of measurement, repeat measurements to detect problems over time may require an expert to return to the same location. We therefore developed an image-guidance system based on ultrasound B-mode images that enables an inexperienced user to position the ultrasound probe at the same site repeatedly in order to acquire a comparable time series of Doppler readings. The system utilizes a bioresorbable fiducial and complementing software composed of the fiducial detection, key points tracking, probe pose estimation, and graphical user interface (GUI) modules. The fiducial is an echogenic marker that is implanted at the surgical site and can be detected and tracked during ultrasound B-mode screening. The key points on the marker can next be used to determine the pose of the ultrasound probe with respect to the marker. The 3D representation of the probe with its position and orientation are then displayed in the GUI for the user guidance. The fiducial detection has been tested on the data sets collected from three animal studies. The pose estimation algorithm was validated by five data sets collected by a UR5 robot. We tested the system on a plastisol phantom and showed that it can detect and track the fiducial marker while displaying the probe pose in real-time.