Displaying 3D radiation dose on endoscopic video for therapeutic assessment and surgical guidance

Cost-function testing methodology for image-based registration of endoscopy to CT images in the head and neck

One of the largest geometric uncertainties in designing radiotherapy treatment plans for squamous cell cancers of the head and neck is contouring the gross tumour volume. We have previously described a method of projecting mucosal disease contours, visible on endoscopy, to volumetrically reconstructed planning CT datasets, using electromagnetic (EM) tracking of a flexible endoscope, enabling rigid registration between endoscopic and CT images. However, to achieve better accuracy for radiotherapy planning, we propose refining this initial registration with image-based registration methods. In this paper, several types of cost functions are evaluated based on accuracy and robustness. Three phantoms and eight clinical cases are used to test each cost function, with initial registration of endoscopy to CT provided by the pose of the flexible endoscope recovered from EM tracking. Cost function classes include: cross correlation, mutual information and gradient methods. For each test case, a ground truth virtual camera pose was first defined by manual registration of anatomical features visible in both real and virtual endoscope images. A new set of evenly spaced fiducial points and a sample contour were created and projected onto the CT image to be used in assessing image registration quality. A new set of 5000 displaced poses was generated by random sampling displacements along each translational and rotational dimension. At each pose, fiducial and contour points in the real image were again projected on the CT image. The cost function, fiducial registration error and contouring error values were then calculated. While all cost functions performed well in select cases, only the normalized gradient field function consistently had registration errors less than 2 mm, which is the accuracy needed if this application of registering mucosal disease identified on optical image to CT images is to be used in the clinical practice of radiation treatment planning. (Registration: ClinicalTrials.gov NCT02704169).

In situ tissue pathology from spatially encoded mass spectrometry classifiers visualized in real time through augmented reality

Integration between a hand-held mass spectrometry desorption probe based on picosecond infrared laser technology (PIRL-MS) and an optical surgical tracking system demonstrates in situ tissue pathology from point-sampled mass spectrometry data. Spatially encoded pathology classifications are displayed at the site of laser sampling as color-coded pixels in an augmented reality video feed of the surgical field of view. This is enabled by two-way communication between surgical navigation and mass spectrometry data analysis platforms through a custom-built interface. Performance of the system was evaluated using murine models of human cancers sampled in situ in the presence of body fluids with a technical pixel error of 1.0 ± 0.2 mm, suggesting a 84% or 92% (excluding one outlier) cancer type classification rate across different molecular models that distinguish cell-lines of each class of breast, brain, head and neck murine models. Further, through end-point immunohistochemical staining for DNA damage, cell death and neuronal viability, spatially encoded PIRL-MS sampling is shown to produce classifiable mass spectral data from living murine brain tissue, with levels of neuronal damage that are comparable to those induced by a surgical scalpel. This highlights the potential of spatially encoded PIRL-MS analysis for in vivo use during neurosurgical applications of cancer type determination or point-sampling in vivo tissue during tumor bed examination to assess cancer removal. The interface developed herein for the analysis and the display of spatially encoded PIRL-MS data can be adapted to other hand-held mass spectrometry analysis probes currently available.

Integration between a hand-held mass spectrometry desorption probe based on picosecond infrared laser technology (PIRL-MS) and an optical surgical tracking system demonstrates in situ tissue pathology from point-sampled mass spectrometry data.

Feasibility study of navigated endoscopy for the placement of high dose rate brachytherapy applicators in the esophagus and lung

PURPOSE

To evaluate the electromagnetic (EM) tracking of endoscopes and applicators as a method of positioning a high dose rate (HDR) luminal applicator.

METHOD

An anatomical phantom consisting of a rigid trachea and flexible esophagus was used to compare applicator placement measurements using EM tracking vs the traditional method using two-dimensional (2D) fluoroscopy and surface skin markers. The phantom included a tumor in the esophagus and several pairs of optically visible points inside the lumen that were used to simulate proximal and distal ends of tumors of varying lengths. The esophagus tumor and lung points were visible on a computed tomography (CT) image of the phantom, which was used as ground truth for the measurements. The EM tracking system was registered to the CT image using fiducial markers. A flexible endoscope was tracked using the EM system and the locations of the proximal and distal ends of the tumor identified and this position recorded. An EM-tracked applicator was then inserted and positioned relative to the tumor markings. The applicator path was mapped using the EM tracking. The gross tumor length (GTL) and the distance between the first dwell position and distal edge of tumor (offset) were measured using the EM tracking and 2D fluoroscopy methods and compared to the same measurements on the CT image.

RESULTS

The errors in GTL using EM tracking were on average -0.5 ± 1.7 mm and 0.7 ± 3.6 mm for esophagus and lung measurements, similar to errors measured using the 2D fluoroscopy method of -0.9 ± 1.2 mm and 3.4 ± 4.4 mm. Offset measurements were slightly larger while using EM tracking relative to the fluoroscopy method but these were not statistically significant.

CONCLUSIONS

Electromagnetic tracking for placement of lumen applicators is feasible and accurate. Tracking of the endoscope that is used to identify the proximal and distal ends of the tumor and of the applicator during insertion generates accurate three-dimensional measurements of the applicator path, GTL and offset. Guiding the placement of intraluminal applicators using EM navigation is potentially attractive for cases with complex insertions, such as those with nonlinear paths or multiple applicator insertions.

The influence of non-rigid anatomy and patient positioning on endoscopy-CT image registration in the head and neck

PURPOSE

To assess the influence of non-rigid anatomy and differences in patient positioning between CT acquisition and endoscopic examination on endoscopy-CT image registration in the head and neck.

METHODS

Radiotherapy planning CTs and 31-35 daily treatment-room CTs were acquired for nineteen patients. Diagnostic CTs were acquired for thirteen of the patients. The surfaces of the airways were segmented on all scans and triangular meshes were created to render virtual endoscopic images with a calibrated pinhole model of an endoscope. The virtual images were used to take projective measurements throughout the meshes, with reference measurements defined as those taken on the planning CTs and test measurements defined as those taken on the daily or diagnostic CTs. The influence of non-rigid anatomy was quantified by 3D distance errors between reference and test measurements on the daily CTs, and the influence of patient positioning was quantified by 3D distance errors between reference and test measurements on the diagnostic CTs. The daily CT measurements were also used to investigate the influences of camera-to-surface distance, surface angle, and the interval of time between scans.

RESULTS

Average errors in the daily CTs were 0.36 ± 0.61 cm in the nasal cavity, 0.58 ± 0.83 cm in the naso- and oropharynx, and 0.47 ± 0.73 cm in the hypopharynx and larynx. Average errors in the diagnostic CTs in those regions were 0.52 ± 0.69 cm, 0.65 ± 0.84 cm, and 0.69 ± 0.90 cm, respectively. All CTs had errors heavily skewed towards 0, albeit with large outliers. Large camera-to-surface distances were found to increase the errors, but the angle at which the camera viewed the surface had no effect. The errors in the Day 1 and Day 15 CTs were found to be significantly smaller than those in the Day 30 CTs (P < 0.05).

CONCLUSIONS

Inconsistencies of patient positioning have a larger influence than non-rigid anatomy on projective measurement errors. In general, these errors are largest when the camera is in the superior pharynx, where it sees large distances and a lot of muscle motion. The errors are larger when the interval of time between CT acquisitions is longer, which suggests that the interval of time between the CT acquisition and the endoscopic examination should be kept short. The median errors found in this study are comparable to acceptable levels of uncertainty in deformable CT registration. Large errors are possible even when image alignment is very good, indicating that projective measurements must be made carefully to avoid these outliers.

The feasibility of endoscopy-CT image registration in the head and neck without prospective endoscope tracking

PURPOSE

Endoscopic examinations are frequently-used procedures for patients with head and neck cancer undergoing radiotherapy, but radiation treatment plans are created on computed tomography (CT) scans. Image registration between endoscopic video and CT could be used to improve treatment planning and analysis of radiation-related normal tissue toxicity. The purpose of this study was to explore the feasibility of endoscopy-CT image registration without prospective physical tracking of the endoscope during the examination.

METHODS

A novel registration technique called Location Search was developed. This technique uses physical constraints on the endoscope's view direction to search for the virtual endoscope coordinates that maximize the similarity between the endoscopic video frame and the virtual endoscopic image. Its performance was tested on phantom and patient images and compared to an established registration technique, Frame-To-Frame Tracking.

RESULTS

In phantoms, Location Search had average registration errors of 0.55 ± 0.60 cm for point measurements and 0.29 ± 0.15 cm for object surface measurements. Frame-To-Frame Tracking achieved similar results on some frames, but it failed on others due to the virtual endoscope becoming lost. This weakness was more pronounced in patients, where Frame-To-Frame tracking could not make it through the nasal cavity. On successful patient video frames, Location Search was able to find endoscope positions with an average distance of 0.98 ± 0.53 cm away from the ground truth positions. However, it failed on many frames due to false similarity matches caused by anatomical structural differences between the endoscopic video and the virtual endoscopic images.

CONCLUSIONS

Endoscopy-CT image registration without prospective physical tracking of the endoscope is possible, but more development is required to achieve an accuracy suitable for clinical translation.