A method for robust segmentation of arbitrarily shaped radiopaque structures in cone-beam CT projections

Real-time nonstandard-shaped gold fiducial marker tracking on x-ray fluoroscopic images for prostate radiotherapy

Objective.The prostate moves in accordance with the movement of surrounding organs. Tumor position can change by ≥3 mm during radiotherapy. Given the difficulties of visualizing the prostate fluoroscopically, fiducial markers are generally implanted into the prostate to monitor its motion during treatment. Recently, internally motion guidance methods of the prostate using a 99.5% gold/0.5% iron flexible notched wire fiducial marker (Gold Anchor® , Naslund Medical AB, Huddinge, Sweden), which requires a 22 gauge needle, has been used. However, because the notched wire can retain its linear shape, acquire a spiral shape, or roll into an irregular ball, detecting it on fluoroscopic images in real-time incurs higher computation costs.Approach.We developed a fiducial tracking algorithm to achieve real-time computation. The marker is detected on the first image frame using a shape filter that employs inter-class variance for the marker likelihood calculated by the filter, focusing on the large difference in densities between the marker and its surroundings. After the second frame, the marker is tracked by adding to the shape filter the similarity to the template cropped from the area around the marker position detected in the first frame. We retrospectively evaluated the algorithm's marker tracking accuracy for ten prostate cases, analyzing two fractions in each case.Main results.Tracking positional accuracy averaged over all patients was 0.13 ± 0.04 mm (mean ± standard deviation, Euclidean distance) and 0.25 ± 0.09 mm (95th percentile). Computation time was 2.82 ± 0.20 ms/frame averaged over all frames.Significance.Our algorithm successfully and stably tracked irregularly-shaped markers in real time.

The first clinical implementation of real-time 6 degree-of-freedom image-guided radiotherapy for liver SABR patients

PURPOSE

Multiple survey results have identified a demand for improved motion management for liver cancer IGRT. Until now, real-time IGRT for liver has been the domain of dedicated and expensive cancer radiotherapy systems. The purpose of this study was to clinically implement and characterise the performance of a novel real-time 6 degree-of-freedom (DoF) IGRT system, Kilovoltage Intrafraction Monitoring (KIM) for liver SABR patients.

METHODS/MATERIALS

The KIM technology segmented gold fiducial markers in intra-fraction x-ray images as a surrogate for the liver tumour and converted the 2D segmented marker positions into a real-time 6DoF tumour position. Fifteen liver SABR patients were recruited and treated with KIM combined with external surrogate guidance at three radiotherapy centres in the TROG 17.03 LARK multi-institutional prospective clinical trial. Patients were either treated in breath-hold or in free breathing using the gating method. The KIM localisation accuracy and dosimetric accuracy achieved with KIM + external surrogate were measured and the results were compared to those with the estimated external surrogate alone.

RESULTS

The KIM localisation accuracy was 0.2±0.9 mm (left-right), 0.3±0.6 mm (superior-inferior) and 1.2±0.8 mm (anterior-posterior) for translations and -0.1◦±0.8◦ (left-right), 0.6◦±1.2◦ (superior-inferior) and 0.1◦±0.9◦ (anterior-posterior) for rotations. The cumulative dose to the GTV with KIM + external surrogate was always within 5% of the plan. In 2 out of 15 patients, >5% dose error would have occurred to the GTV and an organ-at-risk with external surrogate alone.

CONCLUSIONS

This work demonstrates that real-time 6DoF IGRT for liver can be implemented on standard radiotherapy systems to improve treatment accuracy and safety. The observations made during the treatments highlight the potential false assurance of using traditional external surrogates to assess tumour motion in patients and the need for ongoing improvement of IGRT technologies.

A review of artificial intelligence applications for motion tracking in radiotherapy

During radiotherapy, the organs and tumour move as a result of the dynamic nature of the body; this is known as intrafraction motion. Intrafraction motion can result in tumour underdose and healthy tissue overdose, thereby reducing the effectiveness of the treatment while increasing toxicity to the patients. There is a growing appreciation of intrafraction target motion management by the radiation oncology community. Real-time image-guided radiation therapy (IGRT) can track the target and account for the motion, improving the radiation dose to the tumour and reducing the dose to healthy tissue. Recently, artificial intelligence (AI)-based approaches have been applied to motion management and have shown great potential. In this review, four main categories of motion management using AI are summarised: marker-based tracking, markerless tracking, full anatomy monitoring and motion prediction. Marker-based and markerless tracking approaches focus on tracking the individual target throughout the treatment. Full anatomy algorithms monitor for intrafraction changes in the full anatomy within the field of view. Motion prediction algorithms can be used to account for the latencies due to the time for the system to localise, process and act.

Estimating intrafraction tumor motion during fiducial-based liver stereotactic radiotherapy via an iterative closest point (ICP) algorithm

BACKGROUND

Tumor motion may compromise the accuracy of liver stereotactic radiotherapy. In order to carry out a precise planning, estimating liver tumor motion during radiotherapy has received a lot of attention. Previous approach may have difficult to deal with image data corrupted by noise. The iterative closest point (ICP) algorithm is widely used for estimating the rigid registration of three-dimensional point sets when these data were dense or corrupted. In the light of this, our study estimated the three-dimensional (3D) rigid motion of liver tumors during stereotactic liver radiotherapy using reconstructed 3D coordinates of fiducials based on the ICP algorithm.

METHODS

Four hundred ninety-five pairs of orthogonal kilovoltage (KV) images from the CyberKnife stereo imaging system for 12 patients were used in this study. For each pair of images, the 3D coordinates of fiducial markers inside the liver were calculated via geometric derivations. The 3D coordinates were used to calculate the real-time translational and rotational motion of liver tumors around three axes via an ICP algorithm. The residual error was also investigated both with and without rotational correction.

RESULTS

The translational shifts of liver tumors in left-right (LR), anterior-posterior (AP),and superior-inferior (SI) directions were 2.92 ± 1.98 mm, 5.54 ± 3.12 mm, and 16.22 ± 5.86 mm, respectively; the rotational angles in left-right (LR), anterior-posterior (AP), and superior-inferior (SI) directions were 3.95° ± 3.08°, 4.93° ± 2.90°, and 4.09° ± 1.99°, respectively. Rotational correction decreased 3D fiducial displacement from 1.19 ± 0.35 mm to 0.65 ± 0.24 mm (P<0.001).

CONCLUSIONS

The maximum translational movement occurred in the SI direction. Rotational correction decreased fiducial displacements and increased tumor tracking accuracy.

A deep learning framework for automatic detection of arbitrarily shaped fiducial markers in intrafraction fluoroscopic images

PURPOSE

Real-time image-guided adaptive radiation therapy (IGART) requires accurate marker segmentation to resolve three-dimensional (3D) motion based on two-dimensional (2D) fluoroscopic images. Most common marker segmentation methods require prior knowledge of marker properties to construct a template. If marker properties are not known, an additional learning period is required to build the template which exposes the patient to an additional imaging dose. This work investigates a deep learning-based fiducial marker classifier for use in real-time IGART that requires no prior patient-specific data or additional learning periods. The proposed tracking system uses convolutional neural network (CNN) models to segment cylindrical and arbitrarily shaped fiducial markers.

METHODS

The tracking system uses a tracking window approach to perform sliding window classification of each implanted marker. Three cylindrical marker training datasets were generated from phantom kilovoltage (kV) and patient intrafraction images with increasing levels of megavoltage (MV) scatter. The cylindrical shaped marker CNNs were validated on unseen kV fluoroscopic images from 12 fractions of 10 prostate cancer patients with implanted gold fiducials. For the training and validation of the arbitrarily shaped marker CNNs, cone beam computed tomography (CBCT) projection images from ten fractions of seven lung cancer patients with implanted coiled markers were used. The arbitrarily shaped marker CNNs were trained using three patients and the other four unseen patients were used for validation. The effects of full training using a compact CNN (four layers with learnable weights) and transfer learning using a pretrained CNN (AlexNet, eight layers with learnable weights) were analyzed. Each CNN was evaluated using a Precision-Recall curve (PRC), the area under the PRC plot (AUC), and by the calculation of sensitivity and specificity. The tracking system was assessed using the validation data and the accuracy was quantified by calculating the mean error, root-mean-square error (RMSE) and the 1st and 99th percentiles of the error.

RESULTS

The fully trained CNN on the dataset with moderate noise levels had a sensitivity of 99.00% and specificity of 98.92%. Transfer learning of AlexNet resulted in a sensitivity and specificity of 99.42% and 98.13%, respectively, for the same datasets. For the arbitrarily shaped marker CNNs, the sensitivity was 98.58% and specificity was 98.97% for the fully trained CNN. The transfer learning CNN had a sensitivity and specificity of 98.49% and 99.56%, respectively. The CNNs were successfully incorporated into a multiple object tracking system for both cylindrical and arbitrarily shaped markers. The cylindrical shaped marker tracking had a mean RMSE of 1.6 ± 0.2 pixels and 1.3 ± 0.4 pixels in the x- and y-directions, respectively. The arbitrarily shaped marker tracking had a mean RMSE of 3.0 ± 0.5 pixels and 2.2 ± 0.4 pixels in the x- and y-directions, respectively.

CONCLUSION

With deep learning CNNs, high classification performances on unseen patient images were achieved for both cylindrical and arbitrarily shaped markers. Furthermore, the application of CNN models to intrafraction monitoring was demonstrated using a simple tracking system. The results demonstrate that CNN models can be used to track markers without prior knowledge of the marker properties or an additional learning period.

Automatic online and real-time tumour motion monitoring during stereotactic liver treatments on a conventional linac by combined optical and sparse monoscopic imaging with kilovoltage x-rays (COSMIK)

The purpose of this study was to develop, validate and clinically demonstrate fully automatic tumour motion monitoring on a conventional linear accelerator by combined optical and sparse monoscopic imaging with kilovoltage x-rays (COSMIK). COSMIK combines auto-segmentation of implanted fiducial markers in cone-beam computed tomography (CBCT) projections and intra-treatment kV images with simultaneous streaming of an external motion signal. A pre-treatment CBCT is acquired with simultaneous recording of the motion of an external marker block on the abdomen. The 3-dimensional (3D) marker motion during the CBCT is estimated from the auto-segmented positions in the projections and used to optimize an external correlation model (ECM) of internal motion as a function of external motion. During treatment, the ECM estimates the internal motion from the external motion at 20 Hz. KV images are acquired every 3 s, auto-segmented, and used to update the ECM for baseline shifts between internal and external motion. The COSMIK method was validated using Calypso-recorded internal tumour motion with simultaneous camera-recorded external motion for 15 liver stereotactic body radiotherapy (SBRT) patients. The validation included phantom experiments and simulations hereof for 12 fractions and further simulations for 42 fractions. The simulations compared the accuracy of COSMIK with ECM-based monitoring without model updates and with model updates based on stereoscopic imaging as well as continuous kilovoltage intrafraction monitoring (KIM) at 10 Hz without an external signal. Clinical real-time tumour motion monitoring with COSMIK was performed offline for 14 liver SBRT patients (41 fractions) and online for one patient (two fractions). The mean 3D root-mean-square error for the four monitoring methods was 1.61 mm (COSMIK), 2.31 mm (ECM without updates), 1.49 mm (ECM with stereoscopic updates) and 0.75 mm (KIM). COSMIK is the first combined kV/optical real-time motion monitoring method used clinically online on a conventional accelerator. COSMIK gives less imaging dose than KIM and is in addition applicable when the kV imager cannot be deployed such as during non-coplanar fields.

Systematic intrafraction shifts of mediastinal lymph node targets between setup imaging and radiation treatment delivery in lung cancer patients

BACKGROUND AND PURPOSE

Internal target motion results in geometrical uncertainties in lung cancer radiotherapy. In this study, we determined the intrafraction motion and baseline shifts of mediastinal lymph node (LN) targets between setup imaging and treatment delivery.

MATERIAL AND METHODS

Ten lung cancer patients with 2-4 fiducial markers implanted in LN targets received intensity-modulated radiotherapy with a daily setup cone-beam CT (CBCT) scan used for online soft-tissue match on the primary tumor. At a total of 122 fractions, 5 Hz fluoroscopic kV images were acquired orthogonal to the MV treatment beam during treatment delivery. Offline, the 3D trajectory of the markers was determined from their projected trajectory in the CBCT projections and in the intra-treatment kV images. Baseline shifts and changes in the respiratory motion amplitude between CBCT and treatment delivery were determined from the 3D trajectories.

RESULTS

Systematic mean LN baseline shifts of 2.2 mm in the cranial direction (standard deviation (SD): 1.8 mm) and 1.0 mm in the posterior direction (SD: 1.2 mm) occurred between CBCT imaging and treatment delivery. The mean motion amplitudes during CBCT and treatment delivery agreed within 0.2 mm in all directions.

CONCLUSIONS

Systematic cranial and posterior intrafraction baseline shifts between CBCT and treatment delivery were observed for mediastinal LN targets. Intrafraction motion amplitudes were stable.

A Bayesian approach for three-dimensional markerless tumor tracking using kV imaging during lung radiotherapy

The ability to monitor tumor motion without implanted markers can potentially enable broad access to more accurate and precise lung radiotherapy. A major challenge is that kilovoltage (kV) imaging based methods are rarely able to continuously track the tumor due to the inferior tumor visibility on 2D kV images. Another challenge is the estimation of 3D tumor position based on only 2D imaging information. The aim of this work is to address both challenges by proposing a Bayesian approach for markerless tumor tracking for the first time. The proposed approach adopts the framework of the extended Kalman filter, which combines a prediction and measurement steps to make the optimal tumor position update. For each imaging frame, the tumor position is first predicted by a respiratory-correlated model. The 2D tumor position on the kV image is then measured by template matching. Finally, the prediction and 2D measurement are combined based on the 3D distribution of tumor positions in the past 10 s and the estimated uncertainty of template matching. To investigate the clinical feasibility of the proposed method, a total of 13 lung cancer patient datasets were used for retrospective validation, including 11 cone-beam CT scan pairs and two stereotactic ablative body radiotherapy cases. The ground truths for tumor motion were generated from the the 3D trajectories of implanted markers or beacons. The mean, standard deviation, and 95th percentile of the 3D tracking error were found to range from 1.6-2.9 mm, 0.6-1.5 mm, and 2.6-5.8 mm, respectively. Markerless tumor tracking always resulted in smaller errors compared to the standard of care. The improvement was the most pronounced in the superior-inferior (SI) direction, with up to 9.5 mm reduction in the 95th-percentile SI error for patients with  >10 mm 5th-to-95th percentile SI tumor motion. The percentage of errors with 3D magnitude  <5 mm was 96.5% for markerless tumor tracking and 84.1% for the standard of care. The feasibility of 3D markerless tumor tracking has been demonstrated on realistic clinical scenarios for the first time. The clinical implementation of the proposed method will enable more accurate and precise lung radiotherapy using existing hardware and workflow. Future work is focused on the clinical and real-time implementation of this method.

Automated target tracking in kilovoltage images using dynamic templates of fiducial marker clusters

PURPOSE

Implanted fiducial markers are often used in radiotherapy to facilitate accurate visualization and localization of tumors. Typically, such markers are used to aid daily patient positioning and to verify the target's position during treatment. These markers can also provide a wealth of information regarding tumor motion, yet determining their accurate position in thousands of images is often prohibitive. This work introduces a novel, automated method for identifying fiducial markers in planar x-ray imaging.

METHODS

In brief, the method was performed as follows. First, using processed CBCT projection images, an automated routine of reconstruction, forward-projection, tracking, and stabilization generated static templates of the marker cluster at arbitrary viewing angles. Breathing data were then incorporated into the same routine, resulting in dynamic templates dependent on both viewing angle and breathing motion. Finally, marker clusters were tracked using normalized cross correlations between templates (either static or dynamic) and CBCT projection images. To quantify the accuracy of the technique, a phantom study was performed and markers were manually tracked by two users to compare the automated technique against human measurements. Then, 75 pretreatment CBCT scans of 15 pancreatic cancer patients were analyzed to test the automated technique under real-life conditions, including several challenging scenarios for tracking fiducial markers (e.g., extraneous metallic objects, field-of-view limitations, and marker migration).

RESULTS

In phantom and patient studies, for both static and dynamic templates, the method automatically tracked visible marker clusters in 100% of projection images. For scans in which a phantom exhibited 0D, 1D, and 3D motion, the automated technique showed median errors of 39 μm, 53 μm, and 93 μm, respectively. Human precision was worse in comparison; median interobserver differences for single markers and for the averaged coordinates of four markers were 183 μm and 120 μm, respectively. In patient scans, the method was robust against a number of confounding factors. Automated tracking was performed accurately despite the presence of radio-opaque, nonmarker objects (e.g., metallic stents, surgical clips) in five patients. This success was attributed to the distinct appearance of clusters as a whole compared to individual markers. Dynamic templates produced higher cross-correlation scores than static templates in patients whose fiducial marker clusters exhibited considerable deformation or rotation during the breathing cycle. For other patients, no significant difference was seen between dynamic and static templates. Additionally, transient differences in the cross-correlation score identified instances where markers disappeared from view.

CONCLUSIONS

A novel, automated method for producing dynamic templates of fiducial marker clusters has been developed. Production of these templates automatically provides measurements of tumor motion that occurred during the CBCT scan that was used to produce them. Additionally, using these templates with intrafractional images could potentially allow for more robust real-time target tracking in radiotherapy.

A longitudinal four-dimensional computed tomography and cone beam computed tomography dataset for image-guided radiation therapy research in lung cancer

PURPOSE

To describe in detail a dataset consisting of serial four-dimensional computed tomography (4DCT) and 4D cone beam CT (4DCBCT) images acquired during chemoradiotherapy of 20 locally advanced, nonsmall cell lung cancer patients we have collected at our institution and shared publicly with the research community.

ACQUISITION AND VALIDATION METHODS

As part of an NCI-sponsored research study 82 4DCT and 507 4DCBCT images were acquired in a population of 20 locally advanced nonsmall cell lung cancer patients undergoing radiation therapy. All subjects underwent concurrent radiochemotherapy to a total dose of 59.4-70.2 Gy using daily 1.8 or 2 Gy fractions. Audio-visual biofeedback was used to minimize breathing irregularity during all fractions, including acquisition of all 4DCT and 4DCBCT acquisitions in all subjects. Target, organs at risk, and implanted fiducial markers were delineated by a physician in the 4DCT images. Image coordinate system origins between 4DCT and 4DCBCT were manipulated in such a way that the images can be used to simulate initial patient setup in the treatment position. 4DCT images were acquired on a 16-slice helical CT simulator with 10 breathing phases and 3 mm slice thickness during simulation. In 13 of the 20 subjects, 4DCTs were also acquired on the same scanner weekly during therapy. Every day, 4DCBCT images were acquired on a commercial onboard CBCT scanner. An optically tracked external surrogate was synchronized with CBCT acquisition so that each CBCT projection was time stamped with the surrogate respiratory signal through in-house software and hardware tools. Approximately 2500 projections were acquired over a period of 8-10 minutes in half-fan mode with the half bow-tie filter. Using the external surrogate, the CBCT projections were sorted into 10 breathing phases and reconstructed with an in-house FDK reconstruction algorithm. Errors in respiration sorting, reconstruction, and acquisition were carefully identified and corrected.

DATA FORMAT AND USAGE NOTES

4DCT and 4DCBCT images are available in DICOM format and structures through DICOM-RT RTSTRUCT format. All data are stored in the Cancer Imaging Archive (TCIA, http://www.cancerimagingarchive.net/) as collection 4D-Lung and are publicly available.

DISCUSSION

Due to high temporal frequency sampling, redundant (4DCT and 4DCBCT) data at similar timepoints, oversampled 4DCBCT, and fiducial markers, this dataset can support studies in image-guided and image-guided adaptive radiotherapy, assessment of 4D voxel trajectory variability, and development and validation of new tools for image registration and motion management.

Automated patient setup and gating using cone beam computed tomography projections

In radiation therapy, fiducial markers are often implanted near tumors and used for patient positioning and respiratory gating purposes. These markers are then used to manually align the patients by matching the markers in the cone beam computed tomography (CBCT) reconstruction to those in the planning CT. This step is time-intensive and user-dependent, and often results in a suboptimal patient setup. We propose a fully automated, robust method based on dynamic programming (DP) for segmenting radiopaque fiducial markers in CBCT projection images, which are then used to automatically optimize the treatment couch position and/or gating window bounds. The mean of the absolute 2D segmentation error of our DP algorithm is 1.3 ± 1.0 mm for 87 markers on 39 patients. Intrafraction images were acquired every 3 s during treatment at two different institutions. For gated patients from Institution A (8 patients, 40 fractions), the DP algorithm increased the delivery accuracy (96 ± 6% versus 91 ± 11%, p < 0.01) compared to the manual setup using kV fluoroscopy. For non-gated patients from Institution B (6 patients, 16 fractions), the DP algorithm performed similarly (1.5 ± 0.8 mm versus 1.6 ± 0.9 mm, p = 0.48) compared to the manual setup matching the fiducial markers in the CBCT to the mean position. Our proposed automated patient setup algorithm only takes 1-2 s to run, requires no user intervention, and performs as well as or better than the current clinical setup.

Markerless tumor tracking using short kilovoltage imaging arcs for lung image-guided radiotherapy

The ability to monitor tumor motion without implanted markers is clinically advantageous for lung image-guided radiotherapy (IGRT). Existing markerless tracking methods often suffer from overlapping structures and low visibility of tumors on kV projection images. We introduce the short arc tumor tracking (SATT) method to overcome these issues. The proposed method utilizes multiple kV projection images selected from a nine-degree imaging arc to improve tumor localization, and respiratory-correlated 4D cone-beam CT (CBCT) prior knowledge to minimize the effects of overlapping anatomies. The 3D tumor position is solved as an optimization problem with prior knowledge incorporated via regularization. We retrospectively validated SATT on 11 clinical scans from four patients with central tumors. These patients represent challenging scenarios for markerless tumor tracking due to the inferior adjacent contrast. The 3D trajectories of implanted fiducial markers were used as the ground truth for tracking accuracy evaluation. In all cases, the tumors were successfully tracked at all gantry angles. Compared to standard pre-treatment CBCT guidance alone, trajectory errors were significantly smaller with tracking in all cases, and the improvements were the most prominent in the superior-inferior direction. The mean 3D tracking error ranged from 2.2-9.9 mm, which was 0.4-2.6 mm smaller compared to pre-treatment CBCT. In conclusion, we were able to directly track tumors with inferior visibility on kV projection images using SATT. Tumor localization accuracies are significantly better with tracking compared to the current standard of care of lung IGRT. Future work involves the prospective evaluation and clinical implementation of SATT.

Automated marker tracking using noisy X-ray images degraded by the treatment beam

This study demonstrates the feasibility of automated marker tracking for the real-time detection of intrafractional target motion using noisy kilovoltage (kV) X-ray images degraded by the megavoltage (MV) treatment beam. The authors previously introduced the in-line imaging geometry, in which the flat-panel detector (FPD) is mounted directly underneath the treatment head of the linear accelerator. They found that the 121 kVp image quality was severely compromised by the 6 MV beam passing through the FPD at the same time. Specific MV-induced artefacts present a considerable challenge for automated marker detection algorithms. For this study, the authors developed a new imaging geometry by re-positioning the FPD and the X-ray tube. This improved the contrast-to-noise-ratio between 40% and 72% at the 1.2 mAs/image exposure setting. The increase in image quality clearly facilitates the quick and stable detection of motion with the aid of a template matching algorithm. The setup was tested with an anthropomorphic lung phantom (including an artificial lung tumour). In the tumour one or three Calypso beacons were embedded to achieve better contrast during MV radiation. For a single beacon, image acquisition and automated marker detection typically took around 76 ± 6 ms. The success rate was found to be highly dependent on imaging dose and gantry angle. To eliminate possible false detections, the authors implemented a training phase prior to treatment beam irradiation and also introduced speed limits for motion between subsequent images.

Automatic tracking of arbitrarily shaped implanted markers in kilovoltage projection images: a feasibility study

PURPOSE

Certain types of commonly used fiducial markers take on irregular shapes upon implantation in soft tissue. This poses a challenge for methods that assume a predefined shape of markers when automatically tracking such markers in kilovoltage (kV) radiographs. The authors have developed a method of automatically tracking regularly and irregularly shaped markers using kV projection images and assessed its potential for detecting intrafractional target motion during rotational treatment.

METHODS

Template-based matching used a normalized cross-correlation with simplex minimization. Templates were created from computed tomography (CT) images for phantom studies and from end-expiration breath-hold planning CT for patient studies. The kV images were processed using a Sobel filter to enhance marker visibility. To correct for changes in intermarker relative positions between simulation and treatment that can introduce errors in automatic matching, marker offsets in three dimensions were manually determined from an approximately orthogonal pair of kV images. Two studies in anthropomorphic phantom were carried out, one using a gold cylindrical marker representing regular shape, another using a Visicoil marker representing irregular shape. Automatic matching of templates to cone beam CT (CBCT) projection images was performed to known marker positions in phantom. In patient data, automatic matching was compared to manual matching as an approximate ground truth. Positional discrepancy between automatic and manual matching of less than 2 mm was assumed as the criterion for successful tracking. Tracking success rates were examined in kV projection images from 22 CBCT scans of four pancreas, six gastroesophageal junction, and one lung cancer patients. Each patient had at least one irregularly shaped radiopaque marker implanted in or near the tumor. In addition, automatic tracking was tested in intrafraction kV images of three lung cancer patients with irregularly shaped markers during 11 volumetric modulated arc treatments. Purpose-built software developed at our institution was used to create marker templates and track the markers embedded in kV images.

RESULTS

Phantom studies showed mean ± standard deviation measurement uncertainty of automatic registration to be 0.14 ± 0.07 mm and 0.17 ± 0.08 mm for Visicoil and gold cylindrical markers, respectively. The mean success rate of automatic tracking with CBCT projections (11 frames per second, fps) of pancreas, gastroesophageal junction, and lung cancer patients was 100%, 99.1% (range 98%-100%), and 100%, respectively. With intrafraction images (approx. 0.2 fps) of lung cancer patients, the success rate was 98.2% (range 97%-100%), and 94.3% (range 93%-97%) using templates from 1.25 mm and 2.5 mm slice spacing CT scans, respectively. Correction of intermarker relative position was found to improve the success rate in two out of eight patients analyzed.

CONCLUSIONS

The proposed method can track arbitrary marker shapes in kV images using templates generated from a breath-hold CT acquired at simulation. The studies indicate its feasibility for tracking tumor motion during rotational treatment. Investigation of the causes of misregistration suggests that its rate of incidence can be reduced with higher frequency of image acquisition, templates made from smaller CT slice spacing, and correction of changes in intermarker relative positions when they occur.

Automatic tracking of gold seed markers from CBCT image projections in lung and prostate radiotherapy

PURPOSE

To construct a method and software to track gold seed implants in prostate and lung patients undergoing radiotherapy using CBCT image projections.

METHODS

A mathematical model was developed in the MatLab (Mathworks, Natick, USA) environment which uses a combination of discreet cosine transforms and filtering to enhance several edge detection methods for identifying and tracking gold seed fiducial markers in images obtained from Varian (Varian Medical Systems, Palo Alto, USA) and Elekta (Kungstensgatan, Sweden) CBCT projections.

RESULTS

Organ motion was captured for 16 prostate patients and 1 lung patient.

CONCLUSION

Image enhancement and edge detection is capable of automatically tracking markers for up to 98% (Varian) and 79% (Elekta) of CBCT projections for prostate and lung markers however inclusion of excessive bony anatomy (LT and RT LAT) inhibit the ability of the model to accurate determine marker location.

Image quality in thoracic 4D cone-beam CT: a sensitivity analysis of respiratory signal, binning method, reconstruction algorithm, and projection angular spacing

PURPOSE

Respiratory signal, binning method, and reconstruction algorithm are three major controllable factors affecting image quality in thoracic 4D cone-beam CT (4D-CBCT), which is widely used in image guided radiotherapy (IGRT). Previous studies have investigated each of these factors individually, but no integrated sensitivity analysis has been performed. In addition, projection angular spacing is also a key factor in reconstruction, but how it affects image quality is not obvious. An investigation of the impacts of these four factors on image quality can help determine the most effective strategy in improving 4D-CBCT for IGRT.

METHODS

Fourteen 4D-CBCT patient projection datasets with various respiratory motion features were reconstructed with the following controllable factors: (i) respiratory signal (real-time position management, projection image intensity analysis, or fiducial marker tracking), (ii) binning method (phase, displacement, or equal-projection-density displacement binning), and (iii) reconstruction algorithm [Feldkamp-Davis-Kress (FDK), McKinnon-Bates (MKB), or adaptive-steepest-descent projection-onto-convex-sets (ASD-POCS)]. The image quality was quantified using signal-to-noise ratio (SNR), contrast-to-noise ratio, and edge-response width in order to assess noise/streaking and blur. The SNR values were also analyzed with respect to the maximum, mean, and root-mean-squared-error (RMSE) projection angular spacing to investigate how projection angular spacing affects image quality.

RESULTS

The choice of respiratory signals was found to have no significant impact on image quality. Displacement-based binning was found to be less prone to motion artifacts compared to phase binning in more than half of the cases, but was shown to suffer from large interbin image quality variation and large projection angular gaps. Both MKB and ASD-POCS resulted in noticeably improved image quality almost 100% of the time relative to FDK. In addition, SNR values were found to increase with decreasing RMSE values of projection angular gaps with strong correlations (r ≈ -0.7) regardless of the reconstruction algorithm used.

CONCLUSIONS

Based on the authors' results, displacement-based binning methods, better reconstruction algorithms, and the acquisition of even projection angular views are the most important factors to consider for improving thoracic 4D-CBCT image quality. In view of the practical issues with displacement-based binning and the fact that projection angular spacing is not currently directly controllable, development of better reconstruction algorithms represents the most effective strategy for improving image quality in thoracic 4D-CBCT for IGRT applications at the current stage.

Real-time automatic fiducial marker tracking in low contrast cine-MV images

PURPOSE

To develop a real-time automatic method for tracking implanted radiographic markers in low-contrast cine-MV patient images used in image-guided radiation therapy (IGRT).

METHODS

Intrafraction motion tracking using radiotherapy beam-line MV images have gained some attention recently in IGRT because no additional imaging dose is introduced. However, MV images have much lower contrast than kV images, therefore a robust and automatic algorithm for marker detection in MV images is a prerequisite. Previous marker detection methods are all based on template matching or its derivatives. Template matching needs to match object shape that changes significantly for different implantation and projection angle. While these methods require a large number of templates to cover various situations, they are often forced to use a smaller number of templates to reduce the computation load because their methods all require exhaustive search in the region of interest. The authors solve this problem by synergetic use of modern but well-tested computer vision and artificial intelligence techniques; specifically the authors detect implanted markers utilizing discriminant analysis for initialization and use mean-shift feature space analysis for sequential tracking. This novel approach avoids exhaustive search by exploiting the temporal correlation between consecutive frames and makes it possible to perform more sophisticated detection at the beginning to improve the accuracy, followed by ultrafast sequential tracking after the initialization. The method was evaluated and validated using 1149 cine-MV images from two prostate IGRT patients and compared with manual marker detection results from six researchers. The average of the manual detection results is considered as the ground truth for comparisons.

RESULTS

The average root-mean-square errors of our real-time automatic tracking method from the ground truth are 1.9 and 2.1 pixels for the two patients (0.26 mm/pixel). The standard deviations of the results from the 6 researchers are 2.3 and 2.6 pixels. The proposed framework takes about 128 ms to detect four markers in the first MV images and about 23 ms to track these markers in each of the subsequent images.

CONCLUSIONS

The unified framework for tracking of multiple markers presented here can achieve marker detection accuracy similar to manual detection even in low-contrast cine-MV images. It can cope with shape deformations of fiducial markers at different gantry angles. The fast processing speed reduces the image processing portion of the system latency, therefore can improve the performance of real-time motion compensation.

Detection and localization of radiotherapy targets by template matching

Radio opaque fiducials are implanted in tumors for the purpose of tracking the target motion using X-ray projections during radiation therapy dose delivery. In this paper we describe and evaluate a novel method based on template matching for detection and localization of arbitrary shaped fiducials. Segmentation methods are not adequate for these fiducials because their appearance in online X-ray projections can vary greatly as a function of imaging angle. The algorithm is based on using the planning CT image to generate templates that correspond to the imaging angles of the online images. We demonstrate successful tracking of complex shape fiducials in clinical images of lung and abdomen. We also validate the algorithm by comparing the results with a segmentation approach for one case in which the fiducials could be tracked by both methods. We also show how by adaptive thresholding of the match scores, we can control the false detection rate.

Automatic tracking of implanted fiducial markers in cone beam CT projection images

PURPOSE

This paper describes a novel method for simultaneous intrafraction tracking of multiple fiducial markers. Although the proposed method is generic and can be adopted for a number of applications including fluoroscopy based patient position monitoring and gated radiotherapy, the tracking results presented in this paper are specific to tracking fiducial markers in a sequence of cone beam CT projection images.

METHODS

The proposed method is accurate and robust thanks to utilizing the mean shift and random sampling principles, respectively. The performance of the proposed method was evaluated with qualitative and quantitative methods, using data from two pancreatic and one prostate cancer patients and a moving phantom. The ground truth, for quantitative evaluation, was calculated based on manual tracking preformed by three observers.

RESULTS

The average dispersion of marker position error calculated from the tracking results for pancreas data (six markers tracked over 640 frames, 3840 marker identifications) was 0.25 mm (at iscoenter), compared with an average dispersion for the manual ground truth estimated at 0.22 mm. For prostate data (three markers tracked over 366 frames, 1098 marker identifications), the average error was 0.34 mm. The estimated tracking error in the pancreas data was < 1 mm (2 pixels) in 97.6% of cases where nearby image clutter was detected and in 100.0% of cases with no nearby image clutter.

CONCLUSIONS

The proposed method has accuracy comparable to that of manual tracking and, in combination with the proposed batch postprocessing, superior robustness. Marker tracking in cone beam CT (CBCT) projections is useful for a variety of purposes, such as providing data for assessment of intrafraction motion, target tracking during rotational treatment delivery, motion correction of CBCT, and phase sorting for 4D CBCT.

Hybrid MV-kV 3D respiratory motion tracking during radiation therapy with low imaging dose

A novel real-time adaptive MV-kV imaging framework for image-guided radiation therapy is developed to reduce the thoracic and abdominal tumor targeting uncertainty caused by respiration-induced intrafraction motion with ultra-low patient imaging dose. In our method, continuous stereoscopic MV-kV imaging is used at the beginning of a radiation therapy delivery for several seconds to measure the implanted marker positions. After this stereoscopic imaging period, the kV imager is switched off except for the times when no fiducial marker is detected in the cine-MV images. The 3D time-varying marker positions are estimated by combining the MV 2D projection data and the motion correlations between directional components of marker motion established from the stereoscopic imaging period and updated afterwards; in particular, the most likely position is assumed to be the position on the projection line that has the shortest distance to the first principal component line segment constructed from previous trajectory points. An adaptive windowed auto-regressive prediction is utilized to predict the marker position a short time later (310 ms and 460 ms in this study) to allow for tracking system latency. To demonstrate the feasibility and evaluate the accuracy of the proposed method, computer simulations were performed for both arc and fixed-gantry deliveries using 66 h of retrospective tumor motion data from 42 patients treated for thoracic or abdominal cancers. The simulations reveal that using our hybrid approach, a smaller than 1.2 mm or 1.5 mm root-mean-square tracking error can be achieved at a system latency of 310 ms or 460 ms, respectively. Because the kV imaging is only used for a short period of time in our method, extra patient imaging dose can be reduced by an order of magnitude compared to continuous MV-kV imaging, while the clinical tumor targeting accuracy for thoracic or abdominal cancers is maintained. Furthermore, no additional hardware is required with the proposed method.

Robust methods for automatic image-to-world registration in cone-beam CT interventional guidance

PURPOSE

Real-time surgical navigation relies on accurate image-to-world registration to align the coordinate systems of the image and patient. Conventional manual registration can present a workflow bottleneck and is prone to manual error and intraoperator variability. This work reports alternative means of automatic image-to-world registration, each method involving an automatic registration marker (ARM) used in conjunction with C-arm cone-beam CT (CBCT). The first involves a Known-Model registration method in which the ARM is a predefined tool, and the second is a Free-Form method in which the ARM is freely configurable.

METHODS

Studies were performed using a prototype C-arm for CBCT and a surgical tracking system. A simple ARM was designed with markers comprising a tungsten sphere within infrared reflectors to permit detection of markers in both x-ray projections and by an infrared tracker. The Known-Model method exercised a predefined specification of the ARM in combination with 3D-2D registration to estimate the transformation that yields the optimal match between forward projection of the ARM and the measured projection images. The Free-Form method localizes markers individually in projection data by a robust Hough transform approach extended from previous work, backprojected to 3D image coordinates based on C-arm geometric calibration. Image-domain point sets were transformed to world coordinates by rigid-body point-based registration. The robustness and registration accuracy of each method was tested in comparison to manual registration across a range of body sites (head, thorax, and abdomen) of interest in CBCT-guided surgery, including cases with interventional tools in the radiographic scene.

RESULTS

The automatic methods exhibited similar target registration error (TRE) and were comparable or superior to manual registration for placement of the ARM within ∼200 mm of C-arm isocenter. Marker localization in projection data was robust across all anatomical sites, including challenging scenarios involving the presence of interventional tools. The reprojection error of marker localization was independent of the distance of the ARM from isocenter, and the overall TRE was dominated by the configuration of individual fiducials and distance from the target as predicted by theory. The median TRE increased with greater ARM-to-isocenter distance (e.g., for the Free-Form method, TRE increasing from 0.78 mm to 2.04 mm at distances of ∼75 mm and 370 mm, respectively). The median TRE within ∼200 mm distance was consistently lower than that of the manual method (TRE = 0.82 mm). Registration performance was independent of anatomical site (head, thorax, and abdomen). The Free-Form method demonstrated a statistically significant improvement (p = 0.0044) in reproducibility compared to manual registration (0.22 mm versus 0.30 mm, respectively).

CONCLUSIONS

Automatic image-to-world registration methods demonstrate the potential for improved accuracy, reproducibility, and workflow in CBCT-guided procedures. A Free-Form method was shown to exhibit robustness against anatomical site, with comparable or improved TRE compared to manual registration. It was also comparable or superior in performance to a Known-Model method in which the ARM configuration is specified as a predefined tool, thereby allowing configuration of fiducials on the fly or attachment to the patient.

Tracking latency in image-based dynamic MLC tracking with direct image access

PURPOSE

Target tracking is a promising method for motion compensation in radiotherapy. For image-based dynamic multileaf collimator (DMLC) tracking, latency has been shown to be the main contributor to geometrical errors in tracking of respiratory motion, specifically due to slow transfer of image data from the image acquisition system to the tracking system via image file storage on a hard disk. The purpose of the current study was to integrate direct image access with a DMLC tracking system and to quantify the tracking latency of the integrated system for both kV and MV image-based tracking.

METHOD

A DMLC tracking system integrated with a linear accelerator was used for tracking of a motion phantom with an embedded tungsten marker. Real-time target localization was based on x-ray images acquired either with a portal imager or a kV imager mounted orthogonal to the treatment beam. Images were processed directly without intermediate disk access. Continuous portal images and system log files were stored during treatment delivery for detailed offline analysis of the tracking latency.

RESULTS

The mean tracking system latency for kV and MV image-based tracking as function of the imaging interval ΔT(image) increased linearly with ΔT(image) as 148 ms + 0.58 * ΔT(image) (kV) and 162 ms + 1.1 * ΔT(image) (MV). The latency contribution from image acquisition and image transfer for kV image-based tracking was independent on ΔT(image) at 103 ± 14 ms. For MV-based tracking, it increased with ΔT(image) as 124 ms + 0.44 * ΔT(image). For ΔT(image) = 200 ms (5 Hz imaging), the total latency was reduced from 550 ms to 264 ms for kV image-based tracking and from 500 ms to 382 ms for MV image-based tracking as compared to the previously used indirect image transfer via image file storage on a hard disk.

CONCLUSION

kV and MV image-based DMLC tracking was successfully integrated with direct image access. It resulted in substantial tracking latency reductions compared with image-based tracking without direct image access.