Robotically controlled three-dimensional micro-ultrasound for prostate biopsy guidance

PURPOSE

Prostate imaging to guide biopsy remains unsatisfactory, with current solutions suffering from high complexity and poor accuracy and reliability. One novel entrant into this field is micro-ultrasound (microUS), which uses a high-frequency imaging probe to achieve very high spatial resolution, and achieves prostate cancer detection rates equivalent to multiparametric magnetic resonance imaging (mpMRI). However, the ExactVu transrectal microUS probe has a unique geometry that makes it challenging to acquire controlled, repeatable three-dimensional (3D) transrectal ultrasound (TRUS) volumes. We describe the design, fabrication, and validation of a 3D acquisition system that allows for the accurate use of the ExactVu microUS device for volumetric prostate imaging.

METHODS

The design uses a motorized, computer-controlled brachytherapy stepper to rotate the ExactVu transducer about its axis. We perform geometric validation using a phantom with known dimensions and compare performance with magnetic resonance imaging (MRI) using a commercial quality assurance anthropomorphic prostate phantom.

RESULTS

Our geometric validation shows accuracy of 1 mm or less in all three directions, and images of an anthropomorphic phantom qualitatively match those acquired using MRI and show good agreement quantitatively.

CONCLUSION

We describe the first system to acquire robotically controlled 3D microUS images using the ExactVu microUS system. The reconstructed 3D microUS images are accurate, which will allow for future applications of the ExactVu microUS system in prostate specimen and in vivo imaging.

Validation of a surface-based deformable MRI-3D ultrasound image registration algorithm toward clinical implementation for interstitial prostate brachytherapy

PURPOSE

The purpose of this study was to evaluate and clinically implement a deformable surface-based magnetic resonance imaging (MRI) to three-dimensional ultrasound (US) image registration algorithm for prostate brachytherapy (BT) with the aim to reduce operator dependence and facilitate dose escalation to an MRI-defined target.

METHODS AND MATERIALS

Our surface-based deformable image registration (DIR) algorithm first translates and scales to align the US- and MR-defined prostate surfaces, followed by deformation of the MR-defined prostate surface to match the US-defined prostate surface. The algorithm performance was assessed in a phantom using three deformation levels, followed by validation in three retrospective high-dose-rate BT clinical cases. For comparison, manual rigid registration and cognitive fusion by physician were also employed. Registration accuracy was assessed using the Dice similarity coefficient (DSC) and target registration error (TRE) for embedded spherical landmarks. The algorithm was then implemented intraoperatively in a prospective clinical case.

RESULTS

In the phantom, our DIR algorithm demonstrated a mean DSC and TRE of 0.74 ± 0.08 and 0.94 ± 0.49 mm, respectively, significantly improving the performance compared to manual rigid registration with 0.64 ± 0.16 and 1.88 ± 1.24 mm, respectively. Clinical results demonstrated reduced variability compared to the current standard of cognitive fusion by physicians.

CONCLUSIONS

We successfully validated a DIR algorithm allowing for translation of MR-defined target and organ-at-risk contours into the intraoperative environment. Prospective clinical implementation demonstrated the intraoperative feasibility of our algorithm, facilitating targeted biopsies and dose escalation to the MR-defined lesion. This method provides the potential to standardize the registration procedure between physicians, reducing operator dependence.

A power Doppler ultrasound method for improving intraoperative tip localization for visually obstructed needles in interstitial prostate brachytherapy

PURPOSE

High-dose-rate (HDR) interstitial brachytherapy (BT) is a common treatment technique for localized intermediate to high-risk prostate cancer. Transrectal ultrasound (US) imaging is typically used for guiding needle insertion, including localization of the needle tip which is critical for treatment planning. However, image artifacts can limit needle tip visibility in standard brightness (B)-mode US, potentially leading to dose delivery that deviates from the planned dose. To improve intraoperative tip visualization in visually obstructed needles, we propose a power Doppler (PD) US method which utilizes a novel wireless mechanical oscillator, validated in phantom experiments and clinical HDR-BT cases as part of a feasibility clinical trial.

METHODS

Our wireless oscillator contains a DC motor housed in a 3D printed case and is powered by rechargeable battery allowing the device to be operated by one person with no additional equipment required in the operating room. The oscillator end-piece features a cylindrical shape designed for BT applications to fit on top of the commonly used cylindrical needle mandrins. Phantom validation was completed using tissue-equivalent agar phantoms with the clinical US system and both plastic and metal needles. Our PD method was tested using a needle implant pattern matching a standard HDR-BT procedure as well as an implant pattern designed to maximize needle shadowing artifacts. Needle tip localization accuracy was assessed using the clinical method based on ideal reference needles as well as a comparison to computed tomography (CT) as a gold standard. Clinical validation was completed in five patients who underwent standard HDR-BT as part of a feasibility clinical trial. Needle tips positions were identified using B-mode US and PD US with perturbation from our wireless oscillator.

RESULTS

Absolute mean ± standard deviation tip error for B-mode alone, PD alone, and B-mode combined with PD was respectively: 0.3 ± 0.3 mm, 0.6 ± 0.5 mm, and 0.4 ± 0.2 mm for the mock HDR-BT needle implant; 0.8 ± 1.7 mm, 0.4 ± 0.6 mm, and 0.3 ± 0.5 mm for the explicit shadowing implant with plastic needles; and 0.5 ± 0.2 mm, 0.5 ± 0.3 mm, and 0.6 ± 0.2 mm for the explicit shadowing implant with metal needles. The total mean absolute tip error for all five patients in the feasibility clinical trial was 0.9 ± 0.7 mm using B-mode US alone and 0.8 ± 0.5 mm when including PD US, with increased benefit observed for needles classified as visually obstructed.

CONCLUSIONS

Our proposed PD needle tip localization method is easy to implement and requires no modifications or additions to the standard clinical equipment or workflow. We have demonstrated decreased tip localization error and variation for visually obstructed needles in both phantom and clinical cases, including providing the ability to visualize needles previously not visible using B-mode US alone. This method has the potential to improve needle visualization in challenging cases without burdening the clinical workflow, potentially improving treatment accuracy in HDR-BT and more broadly in any minimally invasive needle-based procedure.

Effect of dataset size, image quality, and image type on deep learning-based automatic prostate segmentation in 3D ultrasound

Three-dimensional (3D) transrectal ultrasound (TRUS) is utilized in prostate cancer diagnosis and treatment, necessitating time-consuming manual prostate segmentation. We have previously developed an automatic 3D prostate segmentation algorithm involving deep learning prediction on radially sampled 2D images followed by 3D reconstruction, trained on a large, clinically diverse dataset with variable image quality. As large clinical datasets are rare, widespread adoption of automatic segmentation could be facilitated with efficient 2D-based approaches and the development of an image quality grading method. The complete training dataset of 6761 2D images, resliced from 206 3D TRUS volumes acquired using end-fire and side-fire acquisition methods, was split to train two separate networks using either end-fire or side-fire images. Split datasets were reduced to 1000, 500, 250, and 100 2D images. For deep learning prediction, modified U-Net and U-Net++ architectures were implemented and compared using an unseen test dataset of 40 3D TRUS volumes. A 3D TRUS image quality grading scale with three factors (acquisition quality, artifact severity, and boundary visibility) was developed to assess the impact on segmentation performance. For the complete training dataset, U-Net and U-Net++ networks demonstrated equivalent performance, but when trained using split end-fire/side-fire datasets, U-Net++ significantly outperformed the U-Net. Compared to the complete training datasets, U-Net++ trained using reduced-size end-fire and side-fire datasets demonstrated equivalent performance down to 500 training images. For this dataset, image quality had no impact on segmentation performance for end-fire images but did have a significant effect for side-fire images, with boundary visibility having the largest impact. Our algorithm provided fast (<1.5 s) and accurate 3D segmentations across clinically diverse images, demonstrating generalizability and efficiency when employed on smaller datasets, supporting the potential for widespread use, even when data is scarce. The development of an image quality grading scale provides a quantitative tool for assessing segmentation performance.

A review of the research progress of interventional medical equipment and methods for prostate cancer

BACKGROUND

Prostate cancer is a common disease in men and has a relatively high mortality rate. However, the interventional medical equipment used for prostate biopsy and brachytherapy has always been a social concern.

METHODS

To understand interventional medical equipment for prostate cancer, the structure of manual, semi-automatic and automatic medical equipment were considered as the mainline, while the corresponding research on these structures were the auxiliary lines. The characteristics and corresponding research status have been discussed.

RESULTS

Interventional medical equipment for prostate cancer with different degrees of automation and its characteristics were determined, and the imaging principles and characteristics of computed tomography, transrectal ultrasound and magnetic resonance imaging have been briefly described.

CONCLUSION

Certain feasible research suggestions have been proposed for future development from the perspective of structure, accuracy and safety. These include flexible and compact robot structures, high-precision image recognition and guidance, accurate dose planning and monitoring, real-time imaging monitoring without delay, high-precision needle insertion strategy, master-slave control, virtual reality and remote control.

Enhancement of needle visualization and localization in ultrasound

PURPOSE

This scoping review covers needle visualization and localization techniques in ultrasound, where localization-based approaches mostly aim to compute the needle shaft (and tip) location while potentially enhancing its visibility too.

METHODS

A literature review is conducted on the state-of-the-art techniques, which could be divided into five categories: (1) signal and image processing-based techniques to augment the needle, (2) modifications to the needle and insertion to help with needle-transducer alignment and visibility, (3) changes to ultrasound image formation, (4) motion-based analysis and (5) machine learning.

RESULTS

Advantages, limitations and challenges of representative examples in each of the categories are discussed. Evaluation techniques performed in ex vivo, phantom and in vivo studies are discussed and summarized.

CONCLUSION

Greatest limitation of the majority of the literature is that they rely on original visibility of the needle in the static image. Need for additional/improved apparatus is the greatest limitation toward clinical utility in practice.

SIGNIFICANCE

Ultrasound-guided needle placement is performed in many clinical applications, including biopsies, treatment injections and anesthesia. Despite the wide range and long history of this technique, an ongoing challenge is needle visibility in ultrasound. A robust technique to enhance ultrasonic needle visibility, especially for steeply inserted hand-held needles, and while maintaining clinical utility requirements is needed.

Electromagnetic (EM) catheter path tracking in ultrasound-guided brachytherapy of the breast

PURPOSE

To evaluate a novel navigation system for breast brachytherapy, based on ultrasound (US)-guided catheter needle implantations followed by electromagnetic (EM) tracking of catheter paths.

METHODS

Breast phantoms were produced, containing US-visible tumors. Ultrasound was used to localize the tumor pose and volume within the phantom, followed by planning an optimal catheter pattern through the tumor using navigation software. An electromagnetic (EM)-tracked catheter needle was used to insert the catheters in the desired pattern. The inserted catheters were visualized on a post-implant CT, serving as ground truth. Electromagnetic (EM) tracking and reconstruction of the inserted catheter paths were performed by pulling a flexible EM guidewire through each catheter, performed in two clinical brachytherapy suites. The accuracy of EM catheter tracking was evaluated by calculating the Hausdorff distance between the EM-tracked and CT-based catheter paths. The accuracy and clinical feasibility of EM catheter tracking were also evaluated in three breast cancer patients, performed in a separate experiment room.

RESULTS

A total of 71 catheter needles were implanted into 12 phantoms using US guidance and EM navigation, in an average ± SD time of 8.1 ± 2.9 min. The accuracy of EM catheter tracking was dependent on the brachytherapy suite: 2.0 ± 1.2 mm in suite 1 and 0.6 ± 0.2 mm in suite 2. EM catheter tracking was successfully performed in three breast brachytherapy patients. Catheter tracking typically took less than 5 min and had an average accuracy of 1.7 ± 0.3 mm.

CONCLUSION

Our preliminary results show a potential role for US guidance and EM needle navigation for implantation of catheters for breast brachytherapy. EM catheter tracking can accurately assess the implant geometry in breast brachytherapy patients. This methodology has the potential to evaluate catheter positions directly after the implantation and during the several fractions of the treatment.

Development of a 3D ultrasound guidance system for permanent breast seed implantation

PURPOSE

Permanent breast seed implantation (PBSI) is a promising radiotherapy technique for early-stage breast cancer, completed in a single visit by permanently implanting 103 Pd seeds using needles inserted through a template and guided by two-dimensional (2D) ultrasound (US). However, operator dependence has been highlighted as a limitation of this procedure. Consequently, we propose and have developed an intraoperative guidance system using three-dimensional (3D) US and an instrumented mechanical arm to provide intraoperative 3D imaging and needle template tracking.

METHODS

A mechatronic 3D US scanner reconstructs a 3D image from 150 2D images. A tracked mechanical arm mounted to the scanner locates four fiducial points on the template, registering the template to the 3D image. 3D reconstruction was validated for linear and volumetric measurement accuracy using phantoms of known geometry. In vivo breast US image quality was evaluated in a healthy volunteer. The encoded arm was calibrated and validated using a jig with divots at known locations relative to the scanner and the scanner registered to the 3D US image using intersecting strings in a fluid-filled test jig. Template registration accuracy was assessed using a machined test jig. Tracking accuracy was assessed in a liquid medium by comparing tracked and imaged needle tip positions. Finally, the system was used to guide a mock procedure in a patient-specific phantom and micro-CT imaging used to evaluate its accuracy.

RESULTS

Geometric validation showed median distances within ±1.1% of expected values and volumetric validation showed differences of ≤4.1%. Tracking arm point measurements showed an average error of 0.43 mm and 3D US volume registration showed target registration error ≤0.9 mm. Mean template registration accuracy in each axis of translation/rotation was ≤1.3 mm/1.0°. Mean needle-targeting error was 2.5 mm and 1.6° for needle tips and trajectories, respectively. Mean needle tip and angular errors of the phantom procedure were 2.1 mm and 2.6°. Modeled seed displacement of the phantom procedure showed mean error of 2.6 mm and a maximum of 3.8 mm.

CONCLUSIONS

A 3D US guidance system for PBSI has been developed. Benchtop performance and image quality in volunteer scans are satisfactory. A phantom PBSI procedure was successfully delivered using the system with maximum seed error within dosimetric benchmarks (<5 mm). Translation of the device into the clinic is forthcoming.

Fully Automatic Three-Dimensional Ultrasound Imaging Based on Conventional B-Scan

Robotic ultrasound systems have turned into clinical use over the past few decades, increasing precision and quality of medical operations. In this paper, we propose a fully automatic scanning system for three-dimensional (3-D) ultrasound imaging. A depth camera was first used to obtain the depth data and color data of the tissue surface. Based on the depth image, the 3-D contour of the tissue was rendered and the scan path of ultrasound probe was automatically planned. Following the scan path, a 3-D translating device drove the probe to move on the tissue surface. Simultaneously, the B-scans and their positional information were recorded for subsequent volume reconstruction. In order to stop the scanning process when the pressure on the skin exceeded a preset threshold, two force sensors were attached to the front side of the probe for force measurement. In vitro and in vivo experiments were conducted for assessing the performance of the proposed system. Quantitative results show that the error of volume measurement was less than 1%, indicating that the system is capable of automatic ultrasound scanning and 3-D imaging. It is expected that the proposed system can be well used in clinical practices.

Accuracy and variability of high-dose-rate prostate brachytherapy needle tip localization using live two-dimensional and sagittally reconstructed three-dimensional ultrasound

PURPOSE

To measure the accuracy and variability of manual high-dose-rate (HDR) prostate brachytherapy (BT) needle tip localization using sagittally reconstructed three-dimensional (3D) transrectal ultrasound (TRUS) augmented with live two-dimensional (2D) sagittal TRUS.

METHODS AND MATERIALS

Ten prostate cancer patients underwent HDR-BT during which the sagittally assisted sagittally reconstructed (SASR) segmentation technique was completed in parallel with commercially available sagittally assisted axially reconstructed (SAAR) TRUS for comparison. The SASR technique makes use of live 2D ultrasound intraoperatively and allows needle tip updates using the final 3D image in the absence of image artifacts. These updates were repeated offline twice by two separate users. Needle end-length measurements were used to calculate insertion depth errors (IDEs) for each technique.

RESULTS

Images of 147 needles were analyzed. For the SASR technique, both users were confident in tip positions on the final 3D image within 3 mm for 52% of needles, so these tip positions were updated. For the remaining 48% of needles, the tip positions from the live 2D images were used. This SASR technique enabled the localization of all needles with IDEs within ±3 mm for 84% of needles and IDE range of [-6.2 mm, 5.9 mm], compared with 57% and [-8.1 mm, 7.7 mm] when using the commercially available SAAR technique.

CONCLUSIONS

The SASR technique mitigates the impact of 3D TRUS image artifacts on HDR-BT needle tip localization by incorporating live 2D sagittal TRUS intraoperatively and provides a statistically significant reduction in IDE variance compared with the routine SAAR technique.

Simultaneous automatic segmentation of multiple needles using 3D ultrasound for high-dose-rate prostate brachytherapy

PURPOSE

Sagittally reconstructed 3D (SR3D) ultrasound imaging shows promise for improved needle localization for high-dose-rate prostate brachytherapy (HDR-BT); however, needles must be manually segmented intraoperatively while the patient is anesthetized to create a treatment plan. The purpose of this article was to describe and validate an automatic needle segmentation algorithm designed for HDR-BT, specifically capable of simultaneously segmenting all needles in an HDR-BT implant using a single SR3D image with ~5 mm interneedle spacing.

MATERIALS AND METHODS

The segmentation algorithm involves regularized feature point classification and line trajectory identification based on the randomized 3D Hough transform modified to handle multiple straight needles in a single image simultaneously. Needle tips are identified based on peaks in the derivative of the signal intensity profile along the needle trajectory. For algorithm validation, 12 prostate cancer patients underwent HDR-BT during which SR3D images were acquired with all needles in place. Needles present in each of the 12 images were segmented manually, providing a gold standard for comparison, and using the algorithm. Tip errors were assessed in terms of the 3D Euclidean distance between needle tips, and trajectory error was assessed in terms of 2D distance in the axial plane and angular deviation between trajectories.

RESULTS

In total, 190 needles were investigated. Mean execution time of the algorithm was 11.0 s per patient, or 0.7 s per needle. The algorithm identified 82% and 85% of needle tips with 3D errors ≤3 mm and ≤5 mm, respectively, 91% of needle trajectories with 2D errors in the axial plane ≤3 mm, and 83% of needle trajectories with angular errors ≤3°. The largest tip error component was in the needle insertion direction.

CONCLUSIONS

Previous work has indicated HDR-BT needles may be manually segmented using SR3D images with insertion depth errors ≤3 mm and ≤5 mm for 83% and 92% of needles, respectively. The algorithm shows promise for reducing the time required for the segmentation of straight HDR-BT needles, and future work involves improving needle tip localization performance through improved image quality and modeling curvilinear trajectories.

The utilization of magnetic resonance imaging in the operating room

Online image guidance in the operating room using ultrasound imaging led to the resurgence of prostate brachytherapy in the 1980s. Here we describe the evolution of integrating MRI technology in the brachytherapy suite or operating room. Given the complexity, cost, and inherent safety issues associated with MRI system integration, first steps focused on the computational integration of images rather than systems. This approach has broad appeal given minimal infrastructure costs and efficiencies comparable with standard care workflows. However, many concerns remain regarding accuracy of registration through the course of a brachytherapy procedure. In selected academic institutions, MRI systems have been integrated in or near the brachytherapy suite in varied configurations to improve the precision and quality of treatments. Navigation toolsets specifically adapted to prostate brachytherapy are in development and are reviewed.

Towards Clinically Optimized MRI-guided Surgical Manipulator for Minimally Invasive Prostate Percutaneous Interventions: Constructive Design

This paper undertakes the modular design and development of a minimally invasive surgical manipulator for MRI-guided transperineal prostate interventions. Severe constraints for the MRI-compatibility to hold the minimum artifact on the image quality and dimensions restraint of the bore scanner shadow the design procedure. Regarding the constructive design, the manipulator kinematics has been optimized and the effective analytical needle workspace is developed and followed by proposing the workflow for the manual needle insertion. A study of the finite element analysis is established and utilized to improve the mechanism weaknesses under some inevitable external forces to ensure the minimum structure deformation. The procedure for attaching a sterile plastic drape on the robot manipulator is discussed. The introduced robotic manipulator herein is aimed for the clinically prostate biopsy and brachytherapy applications.

Robotic ultrasound systems in medicine

Robots ultrasound (RUS) can be defined as the combination of ultrasound imaging with a robotic system in medical interventions. With their potential for high precision, dexterity, and repeatability, robots are often uniquely suited for ultrasound integration. Although the field is relatively young, it has already generated a multitude of robotic systems for application in dozens of medical procedures. This paper reviews the robotic ultrasound systems that have been developed over the past two decades and describes their potential impact on modern medicine. The RUS projects reviewed include extracorporeal devices, needle guidance systems, and intraoperative systems.

Characterization of ultrasound elevation beamwidth artifacts for prostate brachytherapy needle insertion

PURPOSE

Ultrasound elevation beamwidth leads to image artifacts and uncertainties in localizing objects (such as a surgical needle) in ultrasound images. The authors examined the clinical significance of errors caused by elevation beamwidth artifacts and imaging parameters in needle insertion procedures.

METHODS

Beveled prostate brachytherapy needles were inserted through all holes of a grid template under real-time transrectal ultrasound (TRUS) guidance. The needle tip position as indicated by the TRUS image was compared to their observed physical location. A new device was developed to measure the ultrasound elevation beamwidth.

RESULTS

Imaging parameters of the TRUS scanner have direct impact on the localization error ranging from 0.5 up to 4 mm. The smallest localization error was observed laterally close to the center of the grid template and axially within the beam's focal zone. Largest localization error occurs laterally around both sides of the grid template and axially within the beam's far field. The authors also found that the localization errors vary with both lateral and elevation offsets.

CONCLUSIONS

The authors found properly adjusting the TRUS imaging settings to lower the ultrasound gain and power effectively minimized the appearance of elevation beamwidth artifacts and in turn reduced the localization errors of the needle tip.