Quantitative Third Harmonic Generation Microscopy for Assessment of Glioma in Human Brain Tissue

Self-Supervised Image Denoising of Third Harmonic Generation Microscopic Images of Human Glioma Tissue by Transformer-Based Blind Spot (TBS) Network

Third harmonic generation (THG) microscopy shows great potential for instant pathology of brain tumor tissue during surgery. However, due to the maximal permitted exposure of laser intensity and inherent noise of the imaging system, the noise level of THG images is relatively high, which affects subsequent feature extraction analysis. Denoising THG images is challenging for modern deep-learning based methods because of the rich morphologies contained and the difficulty in obtaining the noise-free counterparts. To address this, in this work, we propose an unsupervised deep-learning network for denoising of THG images which combines a self-supervised blind spot method and a U-shape Transformer using a dynamic sparse attention mechanism. The experimental results on THG images of human glioma tissue show that our approach exhibits superior denoising performance qualitatively and quantitatively compared with previous methods. Our model achieves an improvement of 2.47-9.50 dB in SNR and 0.37-7.40 dB in CNR, compared to six recent state-of-the-art unsupervised learning models including Neighbor2Neighbor, Blind2Unblind, Self2Self+, ZS-N2N, Noise2Info and SDAP. To achieve an objective evaluation of our model, we also validate our model on public datasets including natural and microscopic images, and our model shows a better denoising performance than several recent unsupervised models such as Neighbor2Neighbor, Blind2Unblind and ZS-N2N. In addition, our model is nearly instant in denoising a THG image, which has the potential for real-time applications of THG microscopy.

Compact portable higher harmonic generation microscopy for the real time assessment of unprocessed thyroid tissue

During thyroid surgery fast and reliable intra-operative pathological feedback has the potential to avoid a two-stage procedure and significantly reduce health care costs in patients undergoing a diagnostic hemithyroidectomy (HT). We explored higher harmonic generation (HHG) microscopy, which combines second harmonic generation (SHG), third harmonic generation (THG), and multiphoton excited autofluorescence (MPEF) for this purpose. With a compact, portable HHG microscope, images of freshly excised healthy tissue, benign nodules (follicular adenoma) and malignant tissue (papillary carcinoma, follicular carcinoma and spindle cell carcinoma) were recorded. The images were generated on unprocessed tissue within minutes and show relevant morphological thyroid structures in good accordance with the histology images. The thyroid follicle architecture, cells, cell nuclei (THG), collagen organization (SHG) and the distribution of thyroglobulin and/or thyroid hormones T3 or T4 (MPEF) could be visualized. We conclude that SHG/THG/MPEF imaging is a promising tool for clinical intraoperative assessment of thyroid tissue.

Roadmap on Label-Free Super-Resolution Imaging

Label-free super-resolution (LFSR) imaging relies on light-scattering processes in nanoscale objects without a need for fluorescent (FL) staining required in super-resolved FL microscopy. The objectives of this Roadmap are to present a comprehensive vision of the developments, the state-of-the-art in this field, and to discuss the resolution boundaries and hurdles which need to be overcome to break the classical diffraction limit of the LFSR imaging. The scope of this Roadmap spans from the advanced interference detection techniques, where the diffraction-limited lateral resolution is combined with unsurpassed axial and temporal resolution, to techniques with true lateral super-resolution capability which are based on understanding resolution as an information science problem, on using novel structured illumination, near-field scanning, and nonlinear optics approaches, and on designing superlenses based on nanoplasmonics, metamaterials, transformation optics, and microsphere-assisted approaches. To this end, this Roadmap brings under the same umbrella researchers from the physics and biomedical optics communities in which such studies have often been developing separately. The ultimate intent of this paper is to create a vision for the current and future developments of LFSR imaging based on its physical mechanisms and to create a great opening for the series of articles in this field.

Advances in Intraoperative Glioma Tissue Sampling and Infiltration Assessment

Gliomas are infiltrative brain tumors that often involve functional tissue. While maximal safe resection is critical for maximizing survival, this is challenged by the difficult intraoperative discrimination between tumor-infiltrated and normal structures. Surgical expertise is essential for identifying safe margins, and while the intraoperative pathological review of frozen tissue is possible, this is a time-consuming task. Advances in intraoperative stimulation mapping have aided surgeons in identifying functional structures and, as such, has become the gold standard for this purpose. However, intraoperative margin assessment lacks a similar consensus. Nonetheless, recent advances in intraoperative imaging techniques and tissue examination methods have demonstrated promise for the accurate and efficient assessment of tumor infiltration and margin delineation within the operating room, respectively. In this review, we describe these innovative technologies that neurosurgeons should be aware of.

A deep learning-based stripe self-correction method for stitched microscopic images

Stitched fluorescence microscope images inevitably exist in various types of stripes or artifacts caused by uncertain factors such as optical devices or specimens, which severely affects the image quality and downstream quantitative analysis. Here, we present a deep learning-based Stripe Self-Correction method, so-called SSCOR. Specifically, we propose a proximity sampling scheme and adversarial reciprocal self-training paradigm that enable SSCOR to utilize stripe-free patches sampled from the stitched microscope image itself to correct their adjacent stripe patches. Comparing to off-the-shelf approaches, SSCOR can not only adaptively correct non-uniform, oblique, and grid stripes, but also remove scanning, bubble, and out-of-focus artifacts, achieving the state-of-the-art performance across different imaging conditions and modalities. Moreover, SSCOR does not require any physical parameter estimation, patch-wise manual annotation, or raw stitched information in the correction process. This provides an intelligent prior-free image restoration solution for microscopists or even microscope companies, thus ensuring more precise biomedical applications for researchers.

Leukocyte differentiation in bronchoalveolar lavage fluids using higher harmonic generation microscopy and deep learning

BACKGROUND

In diseases such as interstitial lung diseases (ILDs), patient diagnosis relies on diagnostic analysis of bronchoalveolar lavage fluid (BALF) and biopsies. Immunological BALF analysis includes differentiation of leukocytes by standard cytological techniques that are labor-intensive and time-consuming. Studies have shown promising leukocyte identification performance on blood fractions, using third harmonic generation (THG) and multiphoton excited autofluorescence (MPEF) microscopy.

OBJECTIVE

To extend leukocyte differentiation to BALF samples using THG/MPEF microscopy, and to show the potential of a trained deep learning algorithm for automated leukocyte identification and quantification.

METHODS

Leukocytes from blood obtained from three healthy individuals and one asthma patient, and BALF samples from six ILD patients were isolated and imaged using label-free microscopy. The cytological characteristics of leukocytes, including neutrophils, eosinophils, lymphocytes, and macrophages, in terms of cellular and nuclear morphology, and THG and MPEF signal intensity, were determined. A deep learning model was trained on 2D images and used to estimate the leukocyte ratios at the image-level using the differential cell counts obtained using standard cytological techniques as reference.

RESULTS

Different leukocyte populations were identified in BALF samples using label-free microscopy, showing distinctive cytological characteristics. Based on the THG/MPEF images, the deep learning network has learned to identify individual cells and was able to provide a reasonable estimate of the leukocyte percentage, reaching >90% accuracy on BALF samples in the hold-out testing set.

CONCLUSIONS

Label-free THG/MPEF microscopy in combination with deep learning is a promising technique for instant differentiation and quantification of leukocytes. Immediate feedback on leukocyte ratios has potential to speed-up the diagnostic process and to reduce costs, workload and inter-observer variations.

Rapid digital pathology of H&E-stained fresh human brain specimens as an alternative to frozen biopsy

BACKGROUND

Hematoxylin and Eosin (H&E)-based frozen section (FS) pathology is presently the global standard for intraoperative tumor assessment (ITA). Preparation of frozen section is labor intensive, which might consume up-to 30 minutes, and is susceptible to freezing artifacts. An FS-alternative technique is thus necessary, which is sectioning-free, artifact-free, fast, accurate, and reliably deployable without machine learning and/or additional interpretation training.

METHODS

We develop a training-free true-H&E Rapid Fresh digital-Pathology (the-RFP) technique which is 4 times faster than the conventional preparation of frozen sections. The-RFP is assisted by a mesoscale Nonlinear Optical Gigascope (mNLOG) platform with a streamlined rapid artifact-compensated 2D large-field mosaic-stitching (rac2D-LMS) approach. A sub-6-minute True-H&E Rapid whole-mount-Soft-Tissue Staining (the-RSTS) protocol is introduced for soft/frangible fresh brain specimens. The mNLOG platform utilizes third harmonic generation (THG) and two-photon excitation fluorescence (TPEF) signals from H and E dyes, respectively, to yield the-RFP images.

RESULTS

We demonstrate the-RFP technique on fresh excised human brain specimens. The-RFP enables optically-sectioned high-resolution 2D scanning and digital display of a 1 cm² area in <120 seconds with 3.6 Gigapixels at a sustained effective throughput of >700 M bits/sec, with zero post-acquisition data/image processing. Training-free blind tests considering 50 normal and tumor-specific brain specimens obtained from 8 participants reveal 100% match to the respective formalin-fixed paraffin-embedded (FFPE)-biopsy outcomes.

CONCLUSIONS

We provide a digital ITA solution: the-RFP, which is potentially a fast and reliable alternative to FS-pathology. With H&E-compatibility, the-RFP eliminates color- and morphology-specific additional interpretation training for a pathologist, and the-RFP-assessed specimen can reliably undergo FFPE-biopsy confirmation.

Mechanotherapy in oncology: Targeting nuclear mechanics and mechanotransduction

Mechanotherapy is proposed as a new option for cancer treatment. Increasing evidence suggests that characteristic differences are present in the nuclear mechanics and mechanotransduction of cancer cells compared with those of normal cells. Recent advances in understanding nuclear mechanics and mechanotransduction provide not only further insights into the process of malignant transformation but also useful references for developing new therapeutic approaches. Herein, we present an overview of the alterations of nuclear mechanics and mechanotransduction in cancer cells and highlight their implications in cancer mechanotherapy.

Fast intraoperative histology-based diagnosis of gliomas with third harmonic generation microscopy and deep learning

Management of gliomas requires an invasive treatment strategy, including extensive surgical resection. The objective of the neurosurgeon is to maximize tumor removal while preserving healthy brain tissue. However, the lack of a clear tumor boundary hampers the neurosurgeon's ability to accurately detect and resect infiltrating tumor tissue. Nonlinear multiphoton microscopy, in particular higher harmonic generation, enables label-free imaging of excised brain tissue, revealing histological hallmarks within seconds. Here, we demonstrate a real-time deep learning-based pipeline for automated glioma image analysis, matching video-rate image acquisition. We used a custom noise detection scheme, and a fully-convolutional classification network, to achieve on average 79% binary accuracy, 0.77 AUC and 0.83 mean average precision compared to the consensus of three pathologists, on a preliminary dataset. We conclude that the combination of real-time imaging and image analysis shows great potential for intraoperative assessment of brain tissue during tumor surgery.

Construction of a high-NFOM multiphoton microscope with large-angle resonant raster scanning

A resonant-scanning multiphoton optical microscope (MPM) with a millimeter-scale field-of-view (FOV) often encounters a poor Nyquist figure-of-merit (NFOM), leading to an aliasing effect owing to limited effective voxel-sampling rate. In this protocol, we provide a design guideline to enable high-NFOM MPM imaging while simultaneously securing a large FOV/digital-resolution ratio and a fast resonant raster-scanning speed. We further provide a free version of our custom acquisition software to assist with a smooth and easy construction process.

For complete details on the use and execution of this protocol, please refer to Borah et al. (2021).

•

Critical guidelines to build a high-NFOM laser-scanning multiphoton optical microscope

•

An optimized optical design for large-angle resonant-galvo raster scanning

•

A simplified electronic design for high-speed raster scanning

•

A free-version of C++ based control and acquisition software (LASERaster⁺)

Intravital deep-tumor single-beam 3-photon, 4-photon, and harmonic microscopy

Three-photon excitation has recently been demonstrated as an effective method to perform intravital microscopy in deep, previously inaccessible regions of the mouse brain. The applicability of 3-photon excitation for deep imaging of other, more heterogeneous tissue types has been much less explored. In this work, we analyze the benefit of high-pulse-energy 1 MHz pulse-repetition-rate infrared excitation near 1300 and 1700 nm for in-depth imaging of tumorous and bone tissue. We show that this excitation regime provides a more than 2-fold increased imaging depth in tumor and bone tissue compared to the illumination conditions commonly used in 2-photon excitation, due to improved excitation confinement and reduced scattering. We also show that simultaneous 3- and 4-photon processes can be effectively induced with a single laser line, enabling the combined detection of blue to far-red fluorescence together with second and third harmonic generation without chromatic aberration, at excitation intensities compatible with live tissue imaging. Finally, we analyze photoperturbation thresholds in this excitation regime and derive setpoints for safe cell imaging. Together, these results indicate that infrared high-pulse-energy low-repetition-rate excitation opens novel perspectives for intravital deep-tissue microscopy of multiple parameters in strongly scattering tissues and organs.

Compact portable multiphoton microscopy reveals histopathological hallmarks of unprocessed lung tumor tissue in real time

During lung cancer operations a rapid and reliable assessment of tumor tissue can reduce operation time and potentially improve patient outcomes. We show that third harmonic generation (THG), second harmonic generation (SHG) and two-photon excited autofluorescence (2PEF) microscopy reveals relevant, histopathological information within seconds in fresh unprocessed human lung samples. We used a compact, portable microscope and recorded images within 1 to 3 seconds using a power of 5 mW. The generated THG/SHG/2PEF images of tumorous and nontumorous tissues are compared with the corresponding standard histology images, to identify alveolar structures and histopathological hallmarks. Cellular structures (tumor cells, macrophages and lymphocytes) (THG), collagen (SHG) and elastin (2PEF) are differentiated and allowed for rapid identification of carcinoid with solid growth pattern, minimally enlarged monomorphic cell nuclei with salt-and-pepper chromatin pattern, and adenocarcinoma with lipidic and micropapillary growth patterns. THG/SHG/2PEF imaging is thus a promising tool for clinical intraoperative assessment of lung tumor tissue.

Automated interpretation of time-lapse quantitative phase image by machine learning to study cellular dynamics during epithelial-mesenchymal transition

SIGNIFICANCE

Machine learning is increasingly being applied to the classification of microscopic data. In order to detect some complex and dynamic cellular processes, time-resolved live-cell imaging might be necessary. Incorporating the temporal information into the classification process may allow for a better and more specific classification.

AIM

We propose a methodology for cell classification based on the time-lapse quantitative phase images (QPIs) gained by digital holographic microscopy (DHM) with the goal of increasing performance of classification of dynamic cellular processes.

APPROACH

The methodology was demonstrated by studying epithelial-mesenchymal transition (EMT) which entails major and distinct time-dependent morphological changes. The time-lapse QPIs of EMT were obtained over a 48-h period and specific novel features representing the dynamic cell behavior were extracted. The two distinct end-state phenotypes were classified by several supervised machine learning algorithms and the results were compared with the classification performed on single-time-point images.

RESULTS

In comparison to the single-time-point approach, our data suggest the incorporation of temporal information into the classification of cell phenotypes during EMT improves performance by nearly 9% in terms of accuracy, and further indicate the potential of DHM to monitor cellular morphological changes.

CONCLUSIONS

Proposed approach based on the time-lapse images gained by DHM could improve the monitoring of live cell behavior in an automated fashion and could be further developed into a tool for high-throughput automated analysis of unique cell behavior.