Automatic detection of brachytherapy seeds in 3D ultrasound images using a convolutional neural network

The Big Bang of Deep Learning in Ultrasound-Guided Surgery: A Review

Ultrasound (US) imaging is a paramount modality in many image-guided surgeries and percutaneous interventions, thanks to its high portability, temporal resolution, and cost-efficiency. However, due to its imaging principles, the US is often noisy and difficult to interpret. Appropriate image processing can greatly enhance the applicability of the imaging modality in clinical practice. Compared with the classic iterative optimization and machine learning (ML) approach, deep learning (DL) algorithms have shown great performance in terms of accuracy and efficiency for US processing. In this work, we conduct a comprehensive review on deep-learning algorithms in the applications of US-guided interventions, summarize the current trends, and suggest future directions on the topic.

Artificial intelligence applications in brachytherapy: A literature review

PURPOSE

Artificial intelligence (AI) has the potential to simplify and optimize various steps of the brachytherapy workflow, and this literature review aims to provide an overview of the work done in this field.

METHODS AND MATERIALS

We conducted a literature search in June 2022 on PubMed, Embase, and Cochrane for papers that proposed AI applications in brachytherapy.

RESULTS

A total of 80 papers satisfied inclusion/exclusion criteria. These papers were categorized as follows: segmentation (24), registration and image processing (6), preplanning (13), dose prediction and treatment planning (11), applicator/catheter/needle reconstruction (16), and quality assurance (10). AI techniques ranged from classical models such as support vector machines and decision tree-based learning to newer techniques such as U-Net and deep reinforcement learning, and were applied to facilitate small steps of a process (e.g., optimizing applicator selection) or even automate the entire step of the workflow (e.g., end-to-end preplanning). Many of these algorithms demonstrated human-level performance and offer significant improvements in speed.

CONCLUSIONS

AI has potential to augment, automate, and/or accelerate many steps of the brachytherapy workflow. We recommend that future studies adhere to standard reporting guidelines. We also stress the importance of using larger sample sizes and reporting results using clinically interpretable measures.

Emerging technologies in brachytherapy

Brachytherapy is a mature treatment modality. The literature is abundant in terms of review articles and comprehensive books on the latest established as well as evolving clinical practices. The intent of this article is to part ways and look beyond the current state-of-the-art and review emerging technologies that are noteworthy and perhaps may drive the future innovations in the field. There are plenty of candidate topics that deserve a deeper look, of course, but with practical limits in this communicative platform, we explore four topics that perhaps is worthwhile to review in detail at this time. First, intensity modulated brachytherapy (IMBT) is reviewed. The IMBT takes advantage ofanisotropicradiation profile generated through intelligent high-density shielding designs incorporated onto sources and applicators such to achieve high quality plans. Second, emerging applications of 3D printing (i.e. additive manufacturing) in brachytherapy are reviewed. With the advent of 3D printing, interest in this technology in brachytherapy has been immense and translation swift due to their potential to tailor applicators and treatments customizable to each individual patient. This is followed by, in third, innovations in treatment planning concerning catheter placement and dwell times where new modelling approaches, solution algorithms, and technological advances are reviewed. And, fourth and lastly, applications of a new machine learning technique, called deep learning, which has the potential to improve and automate all aspects of brachytherapy workflow, are reviewed. We do not expect that all ideas and innovations reviewed in this article will ultimately reach clinic but, nonetheless, this review provides a decent glimpse of what is to come. It would be exciting to monitor as IMBT, 3D printing, novel optimization algorithms, and deep learning technologies evolve over time and translate into pilot testing and sensibly phased clinical trials, and ultimately make a difference for cancer patients. Today's fancy is tomorrow's reality. The future is bright for brachytherapy.

Machine learning and registration for automatic seed localization in 3D US images for prostate brachytherapy

PURPOSE

New radiation therapy protocols, in particular adaptive, focal or boost brachytherapy treatments, require determining precisely the position and orientation of the implanted radioactive seeds from real-time ultrasound (US) images. This is necessary to compare them to the planned one and to adjust automatically the dosimetric plan accordingly for next seeds implantations. The image modality, the small size of the seeds, and the artifacts they produce make it a very challenging problem. The objective of the presented work is to setup and to evaluate a robust and automatic method for seed localization in three-dimensional (3D) US images.

METHODS

The presented method is based on a prelocalization of the needles through which the seeds are injected in the prostate. This prelocalization allows focusing the search on a region of interest (ROI) around the needle tip. Seeds localization starts by binarizing the ROI and removing false positives using, respectively, a Bayesian classifier and a support vector machine (SVM). This is followed by a registration stage using first an iterative closest point (ICP) for localizing the connected set of seeds (named strand) inserted through a needle, and secondly refining each seed position using sum of squared differences (SSD) as a similarity criterion. ICP registers a geometric model of the strand to the candidate voxels while SSD compares an appearance model of a single seed to a subset of the image. The method was evaluated both for 3D images of an Agar-agar phantom and a dataset of clinical 3D images. It was tested on stranded and on loose seeds.

RESULTS

Results on phantom and clinical images were compared with a manual localization giving mean errors of 1.09 ± 0.61 mm on phantom image and 1.44 ± 0.45 mm on clinical images. On clinical images, the mean errors of individual seeds orientation was 4.33 ± 8 . 51 ∘ .

CONCLUSIONS

The proposed algorithm for radioactive seed localization is robust, tested on different US images, accurate, giving small mean error values, and returns the five cylindrical seeds degrees of freedom.

Deep learning-based digitization of prostate brachytherapy needles in ultrasound images

PURPOSE

To develop, and evaluate the performance of, a deep learning-based three-dimensional (3D) convolutional neural network (CNN) artificial intelligence (AI) algorithm aimed at finding needles in ultrasound images used in prostate brachytherapy.

METHODS

Transrectal ultrasound (TRUS) image volumes from 1102 treatments were used to create a clinical ground truth (CGT) including 24422 individual needles that had been manually digitized by medical physicists during brachytherapy procedures. A 3D CNN U-net with 128 × 128 × 128 TRUS image volumes as input was trained using 17215 needle examples. Predictions of voxels constituting a needle were combined to yield a 3D linear function describing the localization of each needle in a TRUS volume. Manual and AI digitizations were compared in terms of the root-mean-square distance (RMSD) along each needle, expressed as median and interquartile range (IQR). The method was evaluated on a data set including 7207 needle examples. A subgroup of the evaluation data set (n = 188) was created, where the needles were digitized once more by a medical physicist (G1) trained in brachytherapy. The digitization procedure was timed.

RESULTS

The RMSD between the AI and CGT was 0.55 (IQR: 0.35-0.86) mm. In the smaller subset, the RMSD between AI and CGT was similar (0.52 [IQR: 0.33-0.79] mm) but significantly smaller (P < 0.001) than the difference of 0.75 (IQR: 0.49-1.20) mm between AI and G1. The difference between CGT and G1 was 0.80 (IQR: 0.48-1.18) mm, implying that the AI performed as well as the CGT in relation to G1. The mean time needed for human digitization was 10 min 11 sec, while the time needed for the AI was negligible.

CONCLUSIONS

A 3D CNN can be trained to identify needles in TRUS images. The performance of the network was similar to that of a medical physicist trained in brachytherapy. Incorporating a CNN for needle identification can shorten brachytherapy treatment procedures substantially.

Prostate brachytherapy intraoperative dosimetry using a combination of radiographic seed localization with a C-arm and deformed ultrasound prostate contours

PURPOSE

The purpose of the study was to assess the feasibility of performing intraoperative dosimetry for permanent prostate brachytherapy by combining transrectal ultrasound (TRUS) and fluoroscopy/cone beam CT [CBCT] images and accounting for the effect of prostate deformation.

METHODS AND MATERIALS

13 patients underwent TRUS and multiview two-dimensional fluoroscopic imaging partway through the implant, as well as repeat fluoroscopic imaging with the TRUS probe inserted and retracted, and finally three-dimensional CBCT imaging at the end of the implant. The locations of all the implanted seeds were obtained from the fluoroscopy/CBCT images and were registered to prostate contours delineated on the TRUS images based on a common subset of seeds identified on both image sets. Prostate contours were also deformed, using a finite-element model, to take into account the effect of the TRUS probe pressure. Prostate dosimetry parameters were obtained for fluoroscopic and CBCT-dosimetry approaches and compared with the standard-of-care Day-0 postimplant CT dosimetry.

RESULTS

High linear correlation (R² > 0.8) was observed in the measured values of prostate D_90%, V_100%, and V_150%, between the two intraoperative dosimetry approaches. The prostate D_90% and V_100% obtained from intraoperative dosimetry methods were in agreement with the postimplant CT dosimetry. Only the prostate V_150% was on average 4.1% (p-value <0.05) higher in the CBCT-dosimetry approach and 6.7% (p-value <0.05) higher in postimplant CT dosimetry compared with the fluoroscopic dosimetry approach. Deformation of the prostate by the ultrasound probe appeared to have a minimal effect on prostate dosimetry.

CONCLUSIONS

The results of this study have shown that both of the proposed dosimetric evaluation approaches have potential for real-time intraoperative dosimetry.