A continuous surface reconstruction method on point cloud captured from a 3D surface photogrammetry system

An evaluation of consumer smartphones for generating bolus and surface mould applicators for radiation oncology

BACKGROUND

The use of Computed Tomography (CT) imaging data to create 3D printable patient-specific devices for radiation oncology purposes is already well established in the literature and has shown to have superior conformity than conventional methods. Using non-ionizing radiation imaging techniques such as photogrammetry or laser scanners in-lieu of a CT scanner presents many desirable benefits including reduced imaging dose and fabrication of the device can be completed prior to simulation. With recent advancements in smartphone-based technology, photographic and LiDAR-based technologies are more readily available than ever before and to a high level of quality. As a result, these non-ionizing radiation imaging methods are now able to generate patient-specific devices that can be acceptable for clinical use.

PURPOSE

In this work, we aim to determine if smartphones can be used by radiation oncologists or other radiation oncology staff to generate bolus or brachytherapy surface moulds instead of conventional CT with equivalent or comparable accuracy.

METHODS

This work involved two separate studies: a phantom and participant study. For the phantom study, a RANDO anthropomorphic phantom (limited to the nose region) was used to generate 3D models based on three different imaging techniques: conventional CT, photogrammetry & LiDAR which were both acquired on a smartphone. Virtual boli were designed in Blender and 3D printed from PLA plastic material. The conformity of each printed boli was assessed by measuring the air gap volume and approximate thickness between the phantom & bolus acquired together on a CT. For the participant study, photographs, and a LiDAR scan of four volunteers were captured using an iPhone 13 Pro™ to assess their feasibility for generating human models. Each virtual 3D model was visually assessed to identify any issues in their reconstruction. The LiDAR models were registered to the photogrammetry models where a distance to agreement analysis was performed to assess their level of similarity. Additionally, a 3D virtual bolus was designed and printed using ABS material from all models to assess their conformity onto the participants skin surface using a verbal feedback method.

RESULTS

The photogrammetry derived bolus showed comparable conformity to the CT derived bolus while the LiDAR derived bolus showed poorer conformity as shown by their respective air gap volume and thickness measurements. The reconstruction quality of both the photogrammetry and LiDAR models of the volunteers was inadequate in regions of facial hair and occlusion, which may lead to clinically unacceptable patient-specific device that are created from these areas. All participants found the photogrammetry 3D printed bolus to conform to their nose region with minimal room to move while three of the four participants found the LiDAR was acceptable and could be positioned comfortably over their entire nose.

CONCLUSIONS

Smartphone-based photogrammetry and LiDAR software show great potential for future use in generating 3D reference models for radiation oncology purposes. Further investigations into whether they can be used to fabricate clinically acceptable patient-specific devices on a larger and more diverse cohort of participants and anatomical locations is required for a thorough validation of their clinical usefulness.

Development and prospective in-patient proof-of-concept validation of a surface photogrammetry + CT-based volumetric motion model for lung radiotherapy

PURPOSE

We develop and validate a motion model that uses real-time surface photogrammetry acquired concurrently with four-dimensional computed tomography (4DCT) to estimate respiration-induced changes within the entire irradiated volume, over arbitrarily many respiratory cycles.

METHODS

A research, couch-mounted, VisionRT (VRT) system was used to acquire optical surface data (15 Hz, ROI = 15 × 20 cm² ) from the thoraco-abdominal surface of a consented lung SBRT patient, concurrently with their standard-of-care 4DCT. The end-exhalation phase from the 4DCT was regarded as reference and for each remaining phase, deformation vector fields (DVFs) with respect to the reference phase were computed. To reduce dimensionality, the first two principal components (PCs) of the matrix of nine DVFs were calculated. In parallel, ten phase-averaged VRT surfaces were created. Surface DVFs and corresponding PCs were computed. A principal least squares regression was used to relate the PCs of surface DVF to those of volume DVFs, establishing a relationship between time-varying surface and the underlying time-varying volume. Proof-of-concept validation was performed during each treatment fraction by concurrently acquiring 30 s time series of real-time surface data and "ground truth" kV fluoroscopic data (FL). A ray-tracing algorithm was used to create a digitally reconstructed fluorograph (DRF), and motion trajectories of high-contrast, soft-tissue, anatomical features in the DRF were compared with those from kV FL.

RESULTS

For five of the six fluoroscopic acquisition sessions, the model out-performed 4DCT in predicting contour Dice coefficient with respect to fluoroscopy-derived contours. Similarly, the model exhibited a marked improvement over 4DCT for patch positions on the diaphragm. Model patch position errors varied from 5 to -15 mm while 4DCT errors ranged between 5 and -22.4 mm. For one fluoroscopic acquisition, a marked change in the a priori internal-external correlation resulted in model errors comparable to those of 4DCT.

CONCLUSIONS

We described the development and a proof-of-concept validation for a volumetric motion model that uses surface photogrammetry to correlate the time-varying thoraco-abdominal surface to the time-varying internal thoraco-abdominal volume. These early results indicate that the proposed approach can result in a marked improvement over 4DCT. While limited by the duration of the fluoroscopic acquisitions as well as the resolution of the acquired images, the DRF-based proof-of-concept technique developed here is model-agnostic, and therefore, has the potential to be used as an in-patient validation tool for other volumetric motion models.

Application of optical photogrammetry in radiation oncology: HDR surface mold brachytherapy

PURPOSE

We propose a novel method of designing surface mold brachytherapy applicators using optical photogrammetry. The accuracy of this technique for the purpose of 3D-printing surface mold brachytherapy applicators is investigated.

METHODS AND MATERIALS

Photogrammetry was used to generate a 3D model of a patient's right arm. The geometric accuracy of the model was evaluated against CT in terms of volume, surface area, and the Hausdorff distance. A surface mold applicator was then 3D printed using this reconstructed model. The accuracy was evaluated by analyzing the displacement and air-gap volumes between the applicator and plaster cast on a CT image. This technique was subsequently applied to generate a 3D-printed applicator of the author's hand directly, as a proof of principle, using only photographic images.

RESULTS

The volume and surface area of the model were within 0.1% and 2.6% of the CT-obtained values, respectively. Using the Hausdorff distance metric, it was determined that 93% of the visible vertices present in the CT-derived model had a matching vertex on the photogrammetry-derived model within 1 mm, indicating a high level of similarity. The maximum displacement between the plaster cast of the patient's arm and the photo-derived 3D-printed applicator was 1.2 mm with a total air-gap volume of approximately 0.05 cm³.

CONCLUSIONS

Photogrammetry has been applied to the task of generating 3D-printed brachytherapy surface mold applicators. The current work demonstrates the feasibility and accuracy of this technique and how it may be incorporated into a 3D-printing brachytherapy workflow.

A robust real-time surface reconstruction method on point clouds captured from a 3D surface photogrammetry system

PURPOSE

To develop a robust and real-time surface reconstruction method on point clouds captured from a 3D surface photogrammetry system.

METHODS

The authors have developed a robust and fast surface reconstruction method on point clouds acquired by the photogrammetry system, without explicitly solving the partial differential equation required by a typical variational approach. Taking advantage of the overcomplete nature of the acquired point clouds, their method solves and propagates a sparse linear relationship from the point cloud manifold to the surface manifold, assuming both manifolds share similar local geometry. With relatively consistent point cloud acquisitions, the authors propose a sparse regression (SR) model to directly approximate the target point cloud as a sparse linear combination from the training set, assuming that the point correspondences built by the iterative closest point (ICP) is reasonably accurate and have residual errors following a Gaussian distribution. To accommodate changing noise levels and/or presence of inconsistent occlusions during the acquisition, the authors further propose a modified sparse regression (MSR) model to model the potentially large and sparse error built by ICP with a Laplacian prior. The authors evaluated the proposed method on both clinical point clouds acquired under consistent acquisition conditions and on point clouds with inconsistent occlusions. The authors quantitatively evaluated the reconstruction performance with respect to root-mean-squared-error, by comparing its reconstruction results against that from the variational method.

RESULTS

On clinical point clouds, both the SR and MSR models have achieved sub-millimeter reconstruction accuracy and reduced the reconstruction time by two orders of magnitude to a subsecond reconstruction time. On point clouds with inconsistent occlusions, the MSR model has demonstrated its advantage in achieving consistent and robust performance despite the introduced occlusions.

CONCLUSIONS

The authors have developed a fast and robust surface reconstruction method on point clouds captured from a 3D surface photogrammetry system, with demonstrated sub-millimeter reconstruction accuracy and subsecond reconstruction time. It is suitable for real-time motion tracking in radiotherapy, with clear surface structures for better quantifications.

Prediction of high-dimensional states subject to respiratory motion: a manifold learning approach

The development of high-dimensional imaging systems in image-guided radiotherapy provides important pathways to the ultimate goal of real-time full volumetric motion monitoring. Effective motion management during radiation treatment usually requires prediction to account for system latency and extra signal/image processing time. It is challenging to predict high-dimensional respiratory motion due to the complexity of the motion pattern combined with the curse of dimensionality. Linear dimension reduction methods such as PCA have been used to construct a linear subspace from the high-dimensional data, followed by efficient predictions on the lower-dimensional subspace. In this study, we extend such rationale to a more general manifold and propose a framework for high-dimensional motion prediction with manifold learning, which allows one to learn more descriptive features compared to linear methods with comparable dimensions. Specifically, a kernel PCA is used to construct a proper low-dimensional feature manifold, where accurate and efficient prediction can be performed. A fixed-point iterative pre-image estimation method is used to recover the predicted value in the original state space. We evaluated and compared the proposed method with a PCA-based approach on level-set surfaces reconstructed from point clouds captured by a 3D photogrammetry system. The prediction accuracy was evaluated in terms of root-mean-squared-error. Our proposed method achieved consistent higher prediction accuracy (sub-millimeter) for both 200 ms and 600 ms lookahead lengths compared to the PCA-based approach, and the performance gain was statistically significant.