Handheld line-scanned dual-axis confocal microscope with pistoned MEMS actuation for flat-field fluorescence imaging

Miniature line-scanned dual-axis confocal microscope for versatile clinical use

A miniature optical-sectioning fluorescence microscope with high sensitivity and resolution would enable non-invasive and real-time tissue inspection, with potential use cases including early disease detection and intraoperative guidance. Previously, we developed a miniature MEMS-based dual-axis confocal (DAC) microscope that enabled video-rate optically sectioned in vivo microscopy of human tissues. However, the device's clinical utility was limited due to a small field of view, a non-adjustable working distance, and a lack of a sterilization strategy. In our latest design, we have made improvements to achieve a 2x increase in the field of view (600 × 300 µm) and an adjustable working distance range of 150 µm over a wide range of excitation/emission wavelengths (488-750 nm), all while maintaining a high frame rate of 15 frames per second (fps). Furthermore, the device is designed to image through a disposable sterile plastic drape for convenient clinical use. We rigorously characterize the performance of the device and show example images of ex vivo tissues to demonstrate the optical performance of our new design, including fixed mouse skin and human prostate, as well as fresh mouse kidney, mouse intestine, and human head and neck surgical specimens with corresponding H&E histology. These improvements will facilitate clinical testing and translation.

Miniature side-view dual axes confocal endomicroscope for repetitive in vivo imaging

A side-view dual axes confocal endomicroscope is demonstrated that can be inserted repetitively in hollow organs of genetically engineered mice for in vivo real-time imaging in horizontal and vertical planes. Near infrared (NIR) excitation at λex = 785 nm was used. A monolithic 3-axis parametric resonance scan mirror was fabricated using micro-electro-mechanical systems (MEMS) technology to perform post-objective scanning in the distal end of a 4.19 mm diameter instrument. Torsional and serpentine springs were designed to "switch" the mode of imaging between vertical and horizontal planes by tuning the actuation frequency. This system demonstrated real-time in-vivo images in horizontal and vertical planes with 310 µm depth and 1.75 and 7.5 µm lateral and axial resolution. Individual cells and discrete mucosal structures could be identified.

In vivo microscopy as an adjunctive tool to guide detection, diagnosis, and treatment

SIGNIFICANCE

There have been numerous academic and commercial efforts to develop high-resolution in vivo microscopes for a variety of clinical use cases, including early disease detection and surgical guidance. While many high-profile studies, commercialized products, and publications have resulted from these efforts, mainstream clinical adoption has been relatively slow other than for a few clinical applications (e.g., dermatology).

AIM

Here, our goals are threefold: (1) to introduce and motivate the need for in vivo microscopy (IVM) as an adjunctive tool for clinical detection, diagnosis, and treatment, (2) to discuss the key translational challenges facing the field, and (3) to propose best practices and recommendations to facilitate clinical adoption.

APPROACH

We will provide concrete examples from various clinical domains, such as dermatology, oral/gastrointestinal oncology, and neurosurgery, to reinforce our observations and recommendations.

RESULTS

While the incremental improvement and optimization of IVM technologies should and will continue to occur, future translational efforts would benefit from the following: (1) integrating clinical and industry partners upfront to define and maintain a compelling value proposition, (2) identifying multimodal/multiscale imaging workflows, which are necessary for success in most clinical scenarios, and (3) developing effective artificial intelligence tools for clinical decision support, tempered by a realization that complete adoption of such tools will be slow.

CONCLUSIONS

The convergence of imaging modalities, academic-industry-clinician partnerships, and new computational capabilities has the potential to catalyze rapid progress and adoption of IVM in the next few decades.

Handheld laser scanning microscope catheter for real-time and in vivo confocal microscopy using a high definition high frame rate Lissajous MEMS mirror

A handheld confocal microscope using a rapid MEMS scanning mirror facilitates real-time optical biopsy for simple cancer diagnosis. Here we report a handheld confocal microscope catheter using high definition and high frame rate (HDHF) Lissajous scanning MEMS mirror. The broad resonant frequency region of the fast axis on the MEMS mirror with a low Q-factor facilitates the flexible selection of scanning frequencies. HDHF Lissajous scanning was achieved by selecting the scanning frequencies with high greatest common divisor (GCD) and high total lobe number. The MEMS mirror was fully packaged into a handheld configuration, which was coupled to a home-built confocal imaging system. The confocal microscope catheter allows fluorescence imaging of in vivo and ex vivo mouse tissues with 30 Hz frame rate and 95.4% fill factor at 256 × 256 pixels image, where the lateral resolution is 4.35 μm and the field-of-view (FOV) is 330 μm × 330 μm. This compact confocal microscope can provide diverse handheld microscopic applications for real-time, on-demand, and in vivo optical biopsy.

Intensity-Mosaic: automatic panorama mosaicking of disordered images with insufficient features

Purpose: Handling low-quality and few-feature medical images is a challenging task in automatic panorama mosaicking. Current mosaicking methods for disordered input images are based on feature point matching, whereas in this case intensity-based registration achieves better performance than feature-point registration methods. We propose a mosaicking method that enables the use of mutual information (MI) registration for mosaicking randomly ordered input images with insufficient features. Approach: Dimensionality reduction is used to map disordered input images into a low dimensional space. Based on the low dimensional representation, the image global correspondence can be recognized efficiently. For adjacent image pairs, we optimize the MI metric for registration. The panorama is then created after image blending. We demonstrate our method on relatively lower-cost handheld devices that acquire images from the retina in vivo, kidney ex vivo, and bladder phantom, all of which contain sparse features. Results: Our method is compared with three baselines: AutoStitch, "dimension reduction + SIFT," and "MI-Only." Our method compared to the first two feature-point based methods exhibits 1.25 (ex vivo microscope dataset) to two times (in vivo retina dataset) rate of mosaic completion, and MI-Only has the lowest complete rate among three datasets. When comparing the subsequent complete mosaics, our target registration errors can be 2.2 and 3.8 times reduced when using the microscopy and bladder phantom datasets. Conclusions: Using dimensional reduction increases the success rate of detecting adjacent images, which makes MI-based registration feasible and narrows the search range of MI optimization. To the best of our knowledge, this is the first mosaicking method that allows automatic stitching of disordered images with intensity-based alignment, which provides more robust and accurate results when there are insufficient features for classic mosaicking methods.

Video-Mosaicked Handheld Dual-Axis Confocal Microscopy of Gliomas: An ex vivo Feasibility Study in Humans

Background

Intraoperative confocal microscopy can enable high-resolution cross-sectional imaging of intact tissues as a non-invasive real-time alternative to gold-standard histology. However, all current means of intraoperative confocal microscopy are hindered by a limited field of view (FOV), presenting a challenge for evaluating gliomas, which are highly heterogeneous.

Objective

This study explored the use of image mosaicking with handheld dual-axis confocal (DAC) microscopy of fresh human glioma specimens.

Methods

In this preliminary technical feasibility study, fresh human glioma specimens from 6 patients were labeled with a fast-acting topical stain (acridine orange) and imaged using a newly developed DAC microscope prototype.

Results

In comparison to individual image frames with small fields of view, mosaicked images from a DAC microscope correlate better with gold-standard H&E-stained histology images, including the ability to visualize gradual transitions from areas of dense cellularity to sparse cellularity within the tumor.

Conclusion

LS-DAC microscopy provides high-resolution, high-contrast images of glioma tissues that agree with corresponding H&E histology. Compared with individual image frames, mosaicked images provide more accurate representations of the overall cytoarchitecture of heterogeneous glioma tissues. Further investigation is needed to evaluate the ability of high-resolution mosaicked microscopy to improve the extent of glioma resection and patient outcomes.

Real-time video mosaicking to guide handheld in vivo microscopy

Handheld and endoscopic optical-sectioning microscopes are being developed for noninvasive screening and intraoperative consultation. Imaging a large extent of tissue is often desired, but miniature in vivo microscopes tend to suffer from limited fields of view. To extend the imaging field during clinical use, we have developed a real-time video mosaicking method, which allows users to efficiently survey larger areas of tissue. Here, we modified a previous post-processing mosaicking method so that real-time mosaicking is possible at >30 frames/second when using a device that outputs images that are 400 × 400 pixels in size. Unlike other real-time mosaicking methods, our strategy can accommodate image rotations and deformations that often occur during clinical use of a handheld microscope. We perform a feasibility study to demonstrate that the use of real-time mosaicking is necessary to enable efficient sampling of a desired imaging field when using a handheld dual-axis confocal microscope.

Toward Quantitative Neurosurgical Guidance With High-Resolution Microscopy of 5-Aminolevulinic Acid-Induced Protoporphyrin IX

Low-power fluorescence microscopy of 5-ALA-induced PpIX has emerged as a valuable intraoperative imaging technology for improving the resection of malignant gliomas. However, current fluorescence imaging tools are not highly sensitive nor quantitative, which limits their effectiveness for optimizing operative decisions near the surgical margins of gliomas, in particular non-enhancing low-grade gliomas. Intraoperative high-resolution optical-sectioning microscopy can potentially serve as a valuable complement to low-power fluorescence microscopy by providing reproducible quantification of tumor parameters at the infiltrative margins of diffuse gliomas. In this forward-looking perspective article, we provide a brief discussion of recent technical advancements, pilot clinical studies, and our vision of the future adoption of handheld optical-sectioning microscopy at the final stages of glioma surgeries to enhance the extent of resection. We list a number of challenges for clinical acceptance, as well as potential strategies to overcome such obstacles for the surgical implementation of these in vivo microscopy techniques.

2D Au-Coated Resonant MEMS Scanner for NIR Fluorescence Intraoperative Confocal Microscope

The electrostatic MEMS scanner plays an important role in the miniaturization of the microscopic imaging system. We have developed a new two-dimensional (2D) parametrically-resonant MEMS scanner with patterned Au coating (>90% reflectivity at an NIR 785-nm wavelength), for a near-infrared (NIR) fluorescence intraoperative confocal microscopic imaging system with a compact form factor. A silicon-on-insulator (SOI)-wafer based dicing-free microfabrication process has been developed for mass-production with high yield. Based on an in-plane comb-drive configuration, the resonant MEMS scanner performs 2D Lissajous pattern scanning with a large mechanical scanning angle (MSA, ±4°) on each axis at low driving voltage (36 V). A large field-of-view (FOV) has been achieved by using a post-objective scanning architecture of the confocal microscope. We have integrated the new MEMS scanner into a custom-made NIR fluorescence intraoperative confocal microscope with an outer diameter of 5.5 mm at its distal-end. Axial scanning has been achieved by using a piezoelectric actuator-based driving mechanism. We have successfully demonstrated ex vivo 2D imaging on human tissue specimens with up to five frames/s. The 2D resonant MEMS scanner can potentially be utilized for many applications, including multiphoton microendoscopy and wide-field endoscopy.