Mapping surgical fields by moving a laser-scanning multimodal scope attached to a robot arm

Multimodal 3D cancer-mimicking optical phantom

Three-dimensional (3D) organ-mimicking phantoms provide realistic imaging environments for testing various aspects of optical systems, including for evaluating new probe designs, characterizing the diagnostic potential of new technologies, and assessing novel image processing algorithms prior to validation in real tissue. We introduce and characterize the use of a new material, Dragon Skin (Smooth-On Inc.), and fabrication technique, air-brushing, for fabrication of a 3D phantom that mimics the appearance of a real organ under multiple imaging modalities. We demonstrate the utility of the material and technique by fabricating the first 3D, hollow bladder phantom with realistic normal and multi-stage pathology features suitable for endoscopic detection using the gold standard imaging technique, white light cystoscopy (WLC), as well as the complementary imaging modalities of optical coherence tomography and blue light cystoscopy, which are aimed at improving the sensitivity and specificity of WLC to bladder cancer detection. The flexibility of the material and technique used for phantom construction allowed for the representation of a wide range of diseased tissue states, ranging from inflammation (benign) to high-grade cancerous lesions. Such phantoms can serve as important tools for trainee education and evaluation of new endoscopic instrumentation.

Axial-Stereo 3-D Optical Metrology for Inner Profile of Pipes Using a Scanning Laser Endoscope

As the rapid progress in the development of optoelectronic components and computational power, 3D optical metrology becomes more and more popular in manufacturing and quality control due to its flexibility and high speed. However, most of the optical metrology methods are limited to external surfaces. This paper proposed a new approach to measure tiny internal 3D surfaces with a scanning fiber endoscope and axial-stereo vision algorithm. A dense, accurate point cloud of internally machined threads was generated to compare with its corresponding X-ray 3D data as ground truth, and the quantification was analyzed by Iterative Closest Points algorithm.

Bound constrained bundle adjustment for reliable 3D reconstruction

Bundle adjustment (BA) is a common estimation algorithm that is widely used in machine vision as the last step in a feature-based three-dimensional (3D) reconstruction algorithm. BA is essentially a non-convex non-linear least-square problem that can simultaneously solve the 3D coordinates of all the feature points describing the scene geometry, as well as the parameters of the camera. The conventional BA takes a parameter either as a fixed value or as an unconstrained variable based on whether the parameter is known or not. In cases where the known parameters are inaccurate but constrained in a range, conventional BA results in an incorrect 3D reconstruction by using these parameters as fixed values. On the other hand, these inaccurate parameters can be treated as unknown variables, but this does not exploit the knowledge of the constraints, and the resulting reconstruction can be erroneous since the BA optimization halts at a dramatically incorrect local minimum due to its non-convexity. In many practical 3D reconstruction applications, unknown variables with range constraints are usually available, such as a measurement with a range of uncertainty or a bounded estimate. Thus to better utilize these pre-known, constrained, but inaccurate parameters, a bound constrained bundle adjustment (BCBA) algorithm is proposed, developed and tested in this study. A scanning fiber endoscope (the camera) is used to capture a sequence of images above a surgery phantom (the object) of known geometry. 3D virtual models are reconstructed based on these images and then compared with the ground truth. The experimental results demonstrate BCBA can achieve a more reliable, rapid, and accurate 3D reconstruction than conventional bundle adjustment.