Hyperspectral multiphoton microscopy for in vivo visualization of multiple, spectrally overlapped fluorescent labels

Supercontinuum-tailoring multicolor imaging reveals spatiotemporal dynamics of heterogeneous tumor evolution

Tumor heterogeneity and tumor evolution contribute to cancer treatment failure. To understand how selective pressures drive heterogeneous tumor evolution, it would be useful to image multiple important components and tumor subclones in vivo. We propose a supercontinuum-tailoring two-photon microscope (SCT-TPM) and realize simultaneous observation of nine fluorophores with a single light beam, breaking through the 'color barrier' of intravital two-photon fluorescence imaging. It achieves excitation multiplexing only by modulating the phase of fiber supercontinuum (SC), allowing to capture rapid events of multiple targets with maintaining precise spatial alignment. We employ SCT-TPM to visualize the spatiotemporal dynamics of heterogeneous tumor evolution under host immune surveillance, particularly the behaviors and interactions of six tumor subclones, immune cells and vascular network, and thus infer the trajectories of tumor progression and clonal competition. SCT-TPM opens up the possibility of tumor lineage tracking and mechanism exploration in living biological systems.

Deep learning-enhanced snapshot hyperspectral confocal microscopy imaging system

Laser-scanning confocal hyperspectral microscopy is a powerful technique to identify the different sample constituents and their spatial distribution in three-dimensional (3D). However, it suffers from low imaging speed because of the mechanical scanning methods. To overcome this challenge, we propose a snapshot hyperspectral confocal microscopy imaging system (SHCMS). It combined coded illumination microscopy based on a digital micromirror device (DMD) with a snapshot hyperspectral confocal neural network (SHCNet) to realize single-shot confocal hyperspectral imaging. With SHCMS, high-contrast 160-bands confocal hyperspectral images of potato tuber autofluorescence can be collected by only single-shot, which is almost 5 times improvement in the number of spectral channels than previously reported methods. Moreover, our approach can efficiently record hyperspectral volumetric imaging due to the optical sectioning capability. This fast high-resolution hyperspectral imaging method may pave the way for real-time highly multiplexed biological imaging.

BINGO: a blind unmixing algorithm for ultra-multiplexing fluorescence images

MOTIVATION

Spectral imaging is often used to observe different objects with multiple fluorescent labels to reveal the development of the biological event. As the number of observed objects increases, the spectral overlap between fluorophores becomes more serious, and obtaining a "pure" picture of each fluorophore becomes a major challenge. Here, we propose a blind spectral unmixing algorithm called BINGO (Blind unmixing via SVD-based Initialization Nmf with project Gradient descent and spare cOnstrain), which can extract all kinds of fluorophores more accurately from highly overlapping multichannel data, even if the spectra of the fluorophores are extremely similar or their fluorescence intensity varies greatly.

RESULTS

BINGO can isolate up to 10 fluorophores from spectral imaging data for a single excitation. nine-color living HeLa cells were visualized distinctly with BINGO. It provides an important algorithmic tool for multiplex imaging studies, especially in intravital imaging. BINGO shows great potential in multicolor imaging for biomedical sciences.

AVAILABILITY AND IMPLEMENTATION

The source code used for this paper is available with the test data at https://github.com/Xinyuan555/BINGO_unmixing.

Spectral origami: an angle-variable, wavelength-selective concept with a highly efficient filter-based sensing

This study demonstrates the concept of an angle-variable compact spectral module. As a key feature, the filter-based module enables highly efficient wavelength-selective light detection by applying the reflective beam path according to the origami example. It was accomplished through inclined mirrors, which allow for different incident angles on the wavelength separating interference filters used in a robust assembly with no moving parts. To experimentally verify the concept, a wavelength range between 550 and 700 nm was detected by 11 spectral channels. These initial results showed the potential to develop easily scalable and application-tailored sensors, which can overcome conventional filter-based sensor approaches that use upright or fixed-angle illumination.

Chromatically Corrected Multicolor Multiphoton Microscopy

Simultaneous imaging of multiple labels in tissues is key to studying complex biological processes. Although strategies for color multiphoton excitation have been established, chromatic aberration remains a major problem when multiple excitation wavelengths are used in a scanning microscope. Chromatic aberration introduces a spatial shift between the foci of beams of different wavelengths that varies across the field of view, severely degrading the performance of color imaging. In this work, we propose an adaptive correction strategy that solves this problem in two-beam microscopy techniques. Axial chromatic aberration is corrected by a refractive phase mask that introduces pure defocus into one beam, while lateral chromatic aberration is corrected by a piezoelectric mirror that dynamically compensates for lateral shifts during scanning. We show that this light-efficient approach allows seamless chromatic correction over the entire field of view of different multiphoton objectives without compromising spatial and temporal resolution and that the effective area for beam-mixing processes can be increased by more than 1 order of magnitude. We illustrate this approach with simultaneous three-color, two-photon imaging of developing zebrafish embryos and fixed Brainbow mouse brain slices over large areas. These results establish a robust and efficient method for chromatically corrected multiphoton imaging.

SpeCamX: mobile app that turns unmodified smartphones into multispectral imagers

We present the development of SpeCamX, a mobile application that enables an unmodified smartphone into a multispectral imager. Multispectral imaging provides detailed spectral information about objects or scenes, but its accessibility has been limited due to its specialized requirements for the device. SpeCamX overcomes this limitation by utilizing the RGB photographs captured by smartphones and converting them into multispectral images spanning a range of 420 to 680 nm without a need for internal modifications or external attachments. The app also includes plugin functions for extracting medical information from the resulting multispectral data cube. In a clinical study, SpeCamX was used to implement an augmented smartphone bilirubinometer, predicting blood bilirubin levels (BBL) with superior performance in accuracy, efficiency and stability compared to default smartphone cameras. This innovative technology democratizes multispectral imaging, making it accessible to a wider audience and opening new possibilities for both medical and non-medical applications.

Compact and robust deep learning architecture for fluorescence lifetime imaging and FPGA implementation

This paper reports a bespoke adder-based deep learning network for time-domain fluorescence lifetime imaging (FLIM). By leveraging thel1-norm extraction method, we propose a 1D Fluorescence Lifetime AdderNet (FLAN) without multiplication-based convolutions to reduce the computational complexity. Further, we compressed fluorescence decays in temporal dimension using a log-scale merging technique to discard redundant temporal information derived as log-scaling FLAN (FLAN+LS). FLAN+LS achieves 0.11 and 0.23 compression ratios compared with FLAN and a conventional 1D convolutional neural network (1D CNN) while maintaining high accuracy in retrieving lifetimes. We extensively evaluated FLAN and FLAN+LS using synthetic and real data. A traditional fitting method and other non-fitting, high-accuracy algorithms were compared with our networks for synthetic data. Our networks attained a minor reconstruction error in different photon-count scenarios. For real data, we used fluorescent beads' data acquired by a confocal microscope to validate the effectiveness of real fluorophores, and our networks can differentiate beads with different lifetimes. Additionally, we implemented the network architecture on a field-programmable gate array (FPGA) with a post-quantization technique to shorten the bit-width, thereby improving computing efficiency. FLAN+LS on hardware achieves the highest computing efficiency compared to 1D CNN and FLAN. We also discussed the applicability of our network and hardware architecture for other time-resolved biomedical applications using photon-efficient, time-resolved sensors.

Two-photon fluorescence imaging using a tunable spectral window based on fiber supercontinuum

Two-photon excitation fluorescence (TPEF) microscopy has evolved into a versatile tool in biological research. However, the multiplexing capability of TPEF microscopy is limited by the narrow spectral bandwidth of the light source. In this study, we apply a photonic crystal fiber in TPEF microscopy to broaden the excitation source bandwidth. We tuned the spectral window using a spatial light modulator as a programmable diffraction grating that was placed behind a prism pair. In addition, we combined a grating pair to compensate for dispersion to improve the two-photon excitation efficiency. The combination of a broad spectrum and a programmable grating enabled fast spectral window tuning rate on a time scale of tens of milliseconds. We demonstrate the performance of our method by imaging live B16 cells labeled with four emission spectrum overlapped fluorescent proteins.

Multi-objective optimization of custom compound prism arrays for multiplexed optical imaging

Compound prism arrays are a powerful, yet underutilized, solution for producing high transmission and customized chromatic dispersion profiles over broad bandwidths, the quality of which is unobtainable with commercially available prisms or diffraction gratings. However, the computational complexity associated with designing these prism arrays presents a barrier to the widespread adoption of their use. Here we introduce customizable prism designer software that facilitates high-speed optimization of compound arrays guided by target specifications for chromatic dispersion linearity and detector geometry. Information theory is utilized such that target parameters can be easily modified through user input to efficiently simulate a broad range of possible prism array designs. We demonstrate the capabilities of the designer software to simulate new prism array designs for multiplexed, hyperspectral microscopy that achieve chromatic dispersion linearity and a 70-90% light transmission over a significant portion of the visible wavelength range (500-820 nm). The designer software is applicable to many optical spectroscopy and spectral microscopy applications-with varying requirements for spectral resolution, light ray deviation, and physical size-that are photon-starved and for which the enhanced transmission of refraction versus diffraction warrants custom optical designs.

Few-cycle pulse compression and white light generation in cascaded multipass cells

We report supercontinuum generation and pulse compression in two stacked multipass cells based on dielectric mirrors. The 230 fs pulses at 1 MHz containing 12 µJ are compressed by a factor of 33 down to 7 fs, corresponding to 1.0 GW peak power and overall transmission of 84%. The source is particularly interesting for such applications as time-resolved angle-resolved photoemission spectroscopy (ARPES), photoemission electron microscopy, and nonlinear spectroscopy.

Intravital imaging to study cancer progression and metastasis

Navigation through the bulk tumour, entry into the blood vasculature, survival in the circulation, exit at distant sites and resumption of proliferation are all steps necessary for tumour cells to successfully metastasize. The ability of tumour cells to complete these steps is highly dependent on the timing and sequence of the interactions that these cells have with the tumour microenvironment (TME), including stromal cells, the extracellular matrix and soluble factors. The TME thus plays a major role in determining the overall metastatic phenotype of tumours. The complexity and cause-and-effect dynamics of the TME cannot currently be recapitulated in vitro or inferred from studies of fixed tissue, and are best studied in vivo, in real time and at single-cell resolution. Intravital imaging (IVI) offers these capabilities, and recent years have been a time of immense growth and innovation in the field. Here we review some of the recent advances in IVI of mammalian models of cancer and describe how IVI is being used to understand cancer progression and metastasis, and to develop novel treatments and therapies. We describe new techniques that allow access to a range of tissue and cancer types, novel fluorescent reporters and biosensors that allow fate mapping and the probing of functional and phenotypic states, and the clinical applications that have arisen from applying these techniques, reporters and biosensors to study cancer. We finish by presenting some of the challenges that remain in the field, how to address them and future perspectives.

Multiphoton intravital microscopy of rodents

Tissues are heterogeneous with respect to cellular and non-cellular components and in the dynamic interactions between these elements. To study the behaviour and fate of individual cells in these complex tissues, intravital microscopy (IVM) techniques such as multiphoton microscopy have been developed to visualize intact and live tissues at cellular and subcellular resolution. IVM experiments have revealed unique insights into the dynamic interplay between different cell types and their local environment, and how this drives morphogenesis and homeostasis of tissues, inflammation and immune responses, and the development of various diseases. This Primer introduces researchers to IVM technologies, with a focus on multiphoton microscopy of rodents, and discusses challenges, solutions and practical tips on how to perform IVM. To illustrate the unique potential of IVM, several examples of results are highlighted. Finally, we discuss data reproducibility and how to handle big imaging data sets.

Denoising multiplexed microscopy images in n-dimensional spectral space

Hyperspectral fluorescence microscopy images of biological specimens frequently contain multiple observations of a sparse set of spectral features spread in space with varying intensity. Here, we introduce a spectral vector denoising algorithm that filters out noise without sacrificing spatial information by leveraging redundant observations of spectral signatures. The algorithm applies an n-dimensional Chebyshev or Fourier transform to cluster pixels based on spectral similarity independent of pixel intensity or location, and a denoising convolution filter is then applied in this spectral space. The denoised image may then undergo spectral decomposition analysis with enhanced accuracy. Tests utilizing both simulated and empirical microscopy data indicate that denoising in 3 to 5-dimensional (3D to 5D) spectral spaces decreases unmixing error by up to 70% without degrading spatial resolution.

Microscopy is better in color: development of a streamlined spectral light path for real-time multiplex fluorescence microscopy

Spectroscopic image data has provided molecular discrimination for numerous fields including: remote sensing, food safety and biomedical imaging. Despite the various technologies for acquiring spectral data, there remains a trade-off when acquiring data. Typically, spectral imaging either requires long acquisition times to collect an image stack with high spectral specificity or acquisition times are shortened at the expense of fewer spectral bands or reduced spatial sampling. Hence, new spectral imaging microscope platforms are needed to help mitigate these limitations. Fluorescence excitation-scanning spectral imaging is one such new technology, which allows more of the emitted signal to be detected than comparable emission-scanning spectral imaging systems. Here, we have developed a new optical geometry that provides spectral illumination for use in excitation-scanning spectral imaging microscope systems. This was accomplished using a wavelength-specific LED array to acquire spectral image data. Feasibility of the LED-based spectral illuminator was evaluated through simulation and benchtop testing and assessment of imaging performance when integrated with a widefield fluorescence microscope. Ray tracing simulations (TracePro) were used to determine optimal optical component selection and geometry. Spectral imaging feasibility was evaluated using a series of 6-label fluorescent slides. The LED-based system response was compared to a previously tested thin-film tunable filter (TFTF)-based system. Spectral unmixing successfully discriminated all fluorescent components in spectral image data acquired from both the LED and TFTF systems. Therefore, the LED-based spectral illuminator provided spectral image data sets with comparable information content so as to allow identification of each fluorescent component. These results provide proof-of-principle demonstration of the ability to combine output from many discrete wavelength LED sources using a double-mirror (Cassegrain style) optical configuration that can be further modified to allow for high speed, video-rate spectral image acquisition. Real-time spectral fluorescence microscopy would allow monitoring of rapid cell signaling processes (i.e., Ca²⁺ and other second messenger signaling) and has potential to be translated to clinical imaging platforms.

Phytoglycogen Encapsulation of Lanthanide-Based Nanoparticles as an Optical Imaging Platform with Therapeutic Potential

Lanthanide-based upconverting nanoparticles (UCNPs) are largely sought-after for biomedical applications ranging from bioimaging to therapy. A straightforward strategy is proposed here using the naturally sourced polymer phytoglycogen to coencapsulate UCNPs with hydrophobic photosensitizers as an optical imaging platform and light-induced therapeutic agents. The resulting multifunctional sub-micrometer-sized luminescent beads are shown to be cytocompatible as carrier materials, which encourages the assessment of their potential in biomedical applications. The loading of UCNPs of various elemental compositions enables multicolor hyperspectral imaging of the UCNP-loaded beads, endowing these materials with the potential to serve as luminescent tags for multiplexed imaging or simultaneous detection of different moieties under near-infrared (NIR) excitation. Coencapsulation of UCNPs and Rose Bengal opens the door for potential application of these microcarriers for collagen crosslinking. Alternatively, coloading UCNPs with Chlorin e6 enables NIR-light triggered generation of reactive oxygen species. Overall, the developed encapsulation methodology offers a straightforward and noncytotoxic strategy yielding water-dispersible UCNPs while preserving their bright and color-tunable upconversion emission that would allow them to fulfill their potential as multifunctional platforms for biomedical applications.

In vivo simultaneous nonlinear absorption Raman and fluorescence (SNARF) imaging of mouse brain cortical structures

Label-free multiphoton microscopy is a powerful platform for biomedical imaging. Recent advancements have demonstrated the capabilities of transient absorption microscopy (TAM) for label-free quantification of hemoglobin and stimulated Raman scattering (SRS) microscopy for pathological assessment of label-free virtual histochemical staining. We propose the combination of TAM and SRS with two-photon excited fluorescence (TPEF) to characterize, quantify, and compare hemodynamics, vessel structure, cell density, and cell identity in vivo between age groups. In this study, we construct a simultaneous nonlinear absorption, Raman, and fluorescence (SNARF) microscope with the highest reported in vivo imaging depth for SRS and TAM at 250–280 μm to enable these multimodal measurements. Using machine learning, we predict capillary-lining cell identities with 90% accuracy based on nuclear morphology and capillary relationship. The microscope and methodology outlined herein provides an exciting route to study several research topics, including neurovascular coupling, blood-brain barrier, and neurodegenerative diseases.

In this study a microscope is constructed that carries out simultaneous nonlinear absorption, Raman, and fluorescence (SNARF). Machine learning is then used to predict capillary-lining cell identities with 90% accuracy based on nuclear morphology and capillary relationship, which in combination with the developed microscope, can provide a means to study several fields such as neurovascular coupling.

Trends in hyperspectral imaging: from environmental and health sensing to structure-property and nano-bio interaction studies

Hyperspectral imaging (HSI) is a technique that allows for the simultaneous acquisition of both spatial and spectral information. While HSI has been known for years in the field of remote sensing, for instance in geology, cultural heritage, or food industries, it recently emerged in the fields of nano- and micromaterials as well as bioimaging and -sensing. Herein, the attractiveness of HSI arises from the suitability for generating knowledge about environment-specific optical properties, such as photoluminescence of optical probes in a biological sample or at a single-crystal/particle level, to be leveraged into better understanding of structure-property relationships and nano-bio interactions, respectively. Moreover, given its excellent spectral resolution, HSI is highly suitable for optical multiplexing in multiple dimensions, as sought after for, e.g., high throughput biological imaging by simultaneous tracking of multiple targets. Overall, HSI is an emerging technique that has the potential to transform analytical approaches from biomedicine to advanced materials research. This Trends Article provides insight into the potential of HSI, highlighting selected examples from well-established fields including environmental monitoring and food quality control to set the stage for the discussion of emerging opportunities at the micro- and nanoscale. Herein, special focus is set on photoluminescent micro- and nanoprobes for health and spectral conversion applications.

Femtosecond optical parametric chirped-pulse amplification in birefringent step-index fiber

We demonstrate an optical parametric chirped-pulse amplifier (OPCPA) that uses birefringence phase matching in a step-index single-mode optical fiber. The OPCPA is pumped with chirped pulses that can be compressed to sub-30-fs duration. The signal (idler) pulses are generated at 905 nm (1270 nm), have 26 nJ (20 nJ) pulse energy, and are compressible to 70 fs duration. The short compressed signal and idler pulse durations are enabled by the broad bandwidth of the pump pulses. Numerical simulations guiding the design are consistent with the experimental results and predict that scaling to higher pulse energies will be possible. Forgoing a photonic crystal fiber for phase-matching offers practical advantages, including allowing energy scaling with mode area.

A Co-Culture-Based Multiparametric Imaging Technique to Dissect Local H2O2 Signals with Targeted HyPer7

Multispectral live-cell imaging is an informative approach that permits detecting biological processes simultaneously in the spatial and temporal domain by exploiting spectrally distinct biosensors. However, the combination of fluorescent biosensors with distinct spectral properties such as different sensitivities, and dynamic ranges can undermine accurate co-imaging of the same analyte in different subcellular locales. We advanced a single-color multiparametric imaging method, which allows simultaneous detection of hydrogen peroxide (H2O2) in multiple cell locales (nucleus, cytosol, mitochondria) using the H2O2 biosensor HyPer7. Co-culturing of endothelial cells stably expressing differentially targeted HyPer7 biosensors paved the way for co-imaging compartmentalized H2O2 signals simultaneously in neighboring cells in a single experimental setup. We termed this approach COMPARE IT, which is an acronym for co-culture-based multiparametric imaging technique. Employing this approach, we detected lower H2O2 levels in mitochondria of endothelial cells compared to the cell nucleus and cytosol under basal conditions. Upon administering exogenous H2O2, the cytosolic and nuclear-targeted probes displayed similarly slow and moderate HyPer7 responses, whereas the mitochondria-targeted HyPer7 signal plateaued faster and reached higher amplitudes. Our results indicate striking differences in mitochondrial H2O2 accumulation of endothelial cells. Here, we present the method's potential as a practicable and informative multiparametric live-cell imaging technique.

Two-photon peak molecular brightness spectra reveal long-wavelength enhancements of multiplexed imaging depth and photostability

The broad use of two-photon microscopy has been enabled in part by Ti:Sapphire femtosecond lasers, which offer a wavelength-tunable source of pulsed excitation. Action spectra have thus been primarily reported for the tunable range of Ti:Sapphire lasers (∼700-1000 nm). However, longer wavelengths offer deeper imaging in tissue via reduced scattering and spectral dips in water absorption, and new generations of pulsed lasers offer wider tunable ranges. We present the peak molecular brightness spectra for eight Alexa Fluor dyes between 700-1300 nm as a first-order surrogate for action spectra measured with an unmodified commercial microscope, which reveal overlapping long-wavelength excitation peaks with potential for multiplexed excitation. We demonstrate simultaneous single-wavelength excitation of six spectrally overlapping fluorophores using either short (∼790 nm) or long (∼1090 nm) wavelengths, and that the newly characterized excitation peaks measured past 1000 nm offer improved photostability and enhanced fidelity of linear spectral unmixing at depth compared to shorter wavelengths.

Synchronously pumped Raman laser for simultaneous degenerate and nondegenerate two-photon microscopy

Two-photon fluorescence microscopy is a nonlinear imaging modality frequently used in deep-tissue imaging applications. A tunable-wavelength multicolor short-pulse source is usually required to excite fluorophores with a wide range of excitation wavelengths. This need is most typically met by solid-state lasers, which are bulky, expensive, and complicated systems. Here, we demonstrate a compact, robust fiber system that generates naturally synchronized femtosecond pulses at 1050 nm and 1200 nm by using a combination of gain-managed and Raman amplification. We image the brain of a mouse and view the blood vessels, neurons, and other cell-like structures using simultaneous degenerate and nondegenerate excitation.

Photo-transformable genetically-encoded optical probes for functional highlighting in vivo

Studying the brain requires knowledge about both structure (i.e., connectome) and function of its constituents (neurons and glia alike). This need has prompted the development of novel tools and techniques, in particular optical techniques to examine cells remotely. Early works (1900's) led to the development of novel cell-staining techniques that, when combined with the use of a very simple light microscope, visualized individual neurons and their subcellular compartments in fixed tissues. Today, highlighting of structure and function can be performed on live cells, notably in vivo, owing to discovery of GFP and subsequent development of genetically encoded fluorescent optical tools. In this review, we focus our attention on a subset of optical biosensors, namely probes whose emission can be modified by light. We designate them photo-transformable genetically encoded probes. The family of photo-transformable probes embraces current probes that undergo photoactivation (PA), photoconversion (PC) or photoswitching (PS). We argue that these are particularly suited for studying multiple features of neurons, such as structure, connectivity and function concomitantly, for functional highlighting of neurons in vivo.

DMD-based hyperspectral microscopy with flexible multiline parallel scanning

As one of the most common hyperspectral microscopy (HSM) techniques, line-scanning HSM is currently utilized in many fields. However, its scanning efficiency is still considered to be inadequate since many biological and chemical processes occur too rapidly to be captured. Accordingly, in this work, a digital micromirror device (DMD) based on microelectromechanical systems (MEMS) is utilized to demonstrate a flexible multiline scanning HSM system. To the best of our knowledge, this is the first line-scanning HSM system in which the number of scanning lines N can be tuned by simply changing the DMD's parallel scanning units according to diverse applications. This brilliant strategy of effortless adjustability relies only on on-chip scanning methods and totally exploits the benefits of parallelization, aiming to achieve nearly an N-time improvement in the detection efficiency and an N-time decrease in the scanning time and data volume compared with the single-line method under the same operating conditions. To validate this, we selected a few samples of different spectral wavebands to perform reflection imaging, transmission imaging, and fluorescence imaging with varying numbers of scanning lines. The results show the great potential of our DMD-based HSM system for the rapid development of cellular biology, material analysis, and so on. In addition, its on-chip scanning process eliminates the inherent microscopic architecture, making the whole system compact, lightweight, portable, and not subject to site constraints.