Computed tomography-based spectral imaging for fluorescence microscopy

High-spatial density snapshot imaging spectrometer enabled by 2-photon fabricated custom fiber bundles

We report on a proof-of-concept snapshot imaging spectrometer developed using an array of optical fibers fabricated with 2-photon polymerization (2PP). The dense input array maps to an output array with engineered void spaces for spectral information. Previously, the development and fabrication of custom fiber arrays for imaging spectrometers have been a complex, time-consuming, and costly process, requiring a semi-manual assembly of commercial components. This work applies an automatic development process based on 2PP additive manufacturing with the Nanoscribe GmbH Quantum X system. The technique allows printing of arbitrary optical quality structures with submicron resolution with less than 5 nm roughness, enabling small core fibers/integrated arrays. Specifically, we developed an array prototype of 40 × 80 with 6-micron pitch at the input and 80-micron pitch at the output. The air-clad fibers had a core diameter of 5 µm. Fabricated optical fiber arrays were incorporated into a prism-based imaging spectrometer system with 48 spectral channels to demonstrate multi-spectral imaging. Imaging of a USAF target and color printed letter C as well as spectral comparisons to a commercial spectrometer were used to validate the performance of the system. These results clearly demonstrate the functionality and potential applications of the 3D-printed fiber-based snapshot imaging spectrometer.

Snapshot spectroscopic microscopy with double spherical slicer mirrors

Snapshot hyperspectral microscopic imaging can obtain the morphological characteristics and chemical specificity of samples simultaneously and instantaneously. We demonstrate a double-slicer spectroscopic microscopy (DSSM) that uses two spherical slicer mirrors to magnify the target image and slice it. These slits are lined up and dispersed, then mapped onto an area-array detector. An anamorphosis unit optimizes the capacity of the limited pixels. With a single shot and image recombination, a data cube can be constructed for sample analysis, and a model of DSSM is simulated. The system covers the spectral range from 500 nm to 642.5 nm with 20 spectral channels. The spatial resolution is 417 nm, and the spectral resolution is 7.5 nm.

On the value of CTIS imagery for neural-network-based classification: a simulation perspective

The computed tomography imaging spectrometer (CTIS) is a snapshot hyperspectral imaging system. Its output is a 2D image of multiplexed spatiospectral projections of the hyperspectral cube of the scene. Traditionally, the 3D cube is reconstructed from this image before further analysis. In this paper, we show that it is possible to learn information directly from the CTIS raw output, by training a neural network to perform binary classification on such images. The use case we study is an agricultural one, as snapshot imagery is used substantially in this field: the detection of apple scab lesions on leaves. To train the network appropriately and to study several degrees of scab infection, we simulated CTIS images of scabbed leaves. This was made possible with a novel CTIS simulator, where special care was taken to preserve realistic pixel intensities compared to true images. To the best of our knowledge, this is the first application of compressed learning on a simulated CTIS system.

Spectral image scanning microscopy

For decades, the confocal microscope has represented one of the dominant imaging systems in biomedical imaging at sub-cellular lengthscales. Recently, however, it has increasingly been replaced by a related, but more powerful successor technique termed image scanning microscopy (ISM). In this article, we present ISM capable of measuring spectroscopic information such as that contained in fluorescence or Raman images. Compared to established confocal spectroscopic imaging systems, our implementation offers similar spectral resolution, but higher spatial resolution and detection efficiency. Color sensitivity is achieved by a grating placed in the detection path in conjunction with a camera collecting both spatial and spectral information. The multidimensional data is processed using multi-view maximum likelihood image reconstruction. Our findings are supported by numerical simulations and experiments on micro beads and double-stained HeLa cells.

In vivo simultaneous multispectral fluorescence imaging with spectral multiplexed volume holographic imaging system

A simultaneous multispectral fluorescence imaging system incorporating multiplexed volume holographic grating (VHG) is developed to acquire multispectral images of an object in one shot. With the multiplexed VHG, the imaging system can provide the distribution and spectral characteristics of multiple fluorophores in the scene. The implementation and performance of the simultaneous multispectral imaging system are presented. Further, the system’s capability in simultaneously obtaining multispectral fluorescence measurements is demonstrated with in vivo experiments on a mouse. The demonstrated imaging system has the potential to obtain multispectral images fluorescence simultaneously.

A review of snapshot multidimensional optical imaging: measuring photon tags in parallel

Multidimensional optical imaging has seen remarkable growth in the past decade. Rather than measuring only the two-dimensional spatial distribution of light, as in conventional photography, multidimensional optical imaging captures light in up to nine dimensions, providing unprecedented information about incident photons' spatial coordinates, emittance angles, wavelength, time, and polarization. Multidimensional optical imaging can be accomplished either by scanning or parallel acquisition. Compared with scanning-based imagers, parallel acquisition-also dubbed snapshot imaging-has a prominent advantage in maximizing optical throughput, particularly when measuring a datacube of high dimensions. Here, we first categorize snapshot multidimensional imagers based on their acquisition and image reconstruction strategies, then highlight the snapshot advantage in the context of optical throughput, and finally we discuss their state-of-the-art implementations and applications.

Hyperspectral optical tomography of intrinsic signals in the rat cortex

We introduce a tomographic approach for three-dimensional imaging of evoked hemodynamic activity, using broadband illumination and diffuse optical tomography (DOT) image reconstruction. Changes in diffuse reflectance in the rat somatosensory cortex due to stimulation of a single whisker were imaged at a frame rate of 5 Hz using a hyperspectral image mapping spectrometer. In each frame, images in 38 wavelength bands from 484 to 652 nm were acquired simultaneously. For data analysis, we developed a hyperspectral DOT algorithm that used the Rytov approximation to quantify changes in tissue concentration of oxyhemoglobin ([Formula: see text]) and deoxyhemoglobin (ctHb) in three dimensions. Using this algorithm, the maximum changes in [Formula: see text] and ctHb were found to occur at [Formula: see text] and [Formula: see text] beneath the surface of the cortex, respectively. Rytov tomographic reconstructions revealed maximal spatially localized increases and decreases in [Formula: see text] and ctHb of [Formula: see text] and [Formula: see text], respectively, with these maximum changes occurring at [Formula: see text] poststimulus. The localized optical signals from the Rytov approximation were greater than those from modified Beer-Lambert, likely due in part to the inability of planar reflectance to account for partial volume effects.

Optical hyperspectral imaging in microscopy and spectroscopy - a review of data acquisition

Rather than simply acting as a photographic camera capturing two-dimensional (x, y) intensity images or a spectrometer acquiring spectra (λ), a hyperspectral imager measures entire three-dimensional (x, y, λ) datacubes for multivariate analysis, providing structural, molecular, and functional information about biological cells or tissue with unprecedented detail. Such data also gives clinical insights for disease diagnosis and treatment. We summarize the principles underpinning this technology, highlight its practical implementation, and discuss its recent applications at microscopic to macroscopic scales. Datacube acquisition strategies in hyperspectral imaging x, y, spatial coordinates; λ, wavelength.

Optical hyperspectral imaging in microscopy and spectroscopy - a review of data acquisition

Rather than simply acting as a photographic camera capturing two-dimensional (x, y) intensity images or a spectrometer acquiring spectra (λ), a hyperspectral imager measures entire three-dimensional (x, y, λ) datacubes for multivariate analysis, providing structural, molecular, and functional information about biological cells or tissue with unprecedented detail. Such data also gives clinical insights for disease diagnosis and treatment. We summarize the principles underpinning this technology, highlight its practical implementation, and discuss its recent applications at microscopic to macroscopic scales. Datacube acquisition strategies in hyperspectral imaging x, y, spatial coordinates; λ, wavelength.

Polarization encoded color camera

Digital cameras would be colorblind if they did not have pixelated color filters integrated into their image sensors. Integration of conventional fixed filters, however, comes at the expense of an inability to modify the camera's spectral properties. Instead, we demonstrate a micropolarizer-based camera that can reconfigure its spectral response. Color is encoded into a linear polarization state by a chiral dispersive element and then read out in a single exposure. The polarization encoded color camera is capable of capturing three-color images at wavelengths spanning the visible to the near infrared.

Snapshot 3D optical coherence tomography system using image mapping spectrometry

A snapshot 3-Dimensional Optical Coherence Tomography system was developed using Image Mapping Spectrometry. This system can give depth information (Z) at different spatial positions (XY) within one camera integration time to potentially reduce motion artifact and enhance throughput. The current (x,y,λ) datacube of (85×356×117) provides a 3D visualization of sample with 400 μm depth and 13.4 μm in transverse resolution. Axial resolution of 16.0 μm can also be achieved in this proof-of-concept system. We present an analysis of the theoretical constraints which will guide development of future systems with increased imaging depth and improved axial and lateral resolutions.

Fast model-based multispectral imaging using nonnegative principal component analysis

Estimation of the spectral reflectance of a scene is a critical problem in image processing and computer vision applications. Model-based multispectral imaging, one of the spectral reflectance estimation methods, can effectively reconstruct the full spectrum using a small number of camera shots. However, it is based on iterative optimization and, thus, is computationally too intensive. In this Letter, we modify the iterative optimization problem to a closed-form problem using nonnegative principal component analysis. The proposed method can substantially reduce the computational cost while maintaining the accuracy.

Quantitative comparison between full-spectrum and filter-based imaging in hyperspectral fluorescence microscopy

We implement a filterless illumination scheme on a hyperspectral fluorescence microscope to achieve full-range spectral imaging. The microscope employs polarisation filtering, spatial filtering and spectral unmixing filtering to replace the role of traditional filters. Quantitative comparisons between full-spectrum and filter-based microscopy are provided in the context of signal dynamic range and accuracy of measured fluorophores' emission spectra. To show potential applications, a five-colour cell immunofluorescence imaging experiment is theoretically simulated. Simulation results indicate that the use of proposed full-spectrum imaging technique may result in three times improvement in signal dynamic range compared to that can be achieved in the filter-based imaging.

Depth-resolved image mapping spectrometer (IMS) with structured illumination

We present a depth-resolved Image Mapping Spectrometer (IMS) which is capable of acquiring 4D (x, y, z, λ) datacubes. Optical sectioning is implemented by structured illumination. The device's spectral imaging performance is demonstrated in a multispectral microsphere and mouse kidney tissue fluorescence imaging experiment. We also compare quantitatively the depth-resolved IMS with a hyperspectral confocal microscope (HCM) in a standard fluorescent bead imaging experiment. The comparison results show that despite the use of a light source with four orders of magnitude lower intensity in the IMS than that in the HCM, the image signal-to-noise ratio acquired by the IMS is 2.6 times higher than that achieved by the equivalent confocal approach.

Real-time snapshot hyperspectral imaging endoscope

Hyperspectral imaging has tremendous potential to detect important molecular biomarkers of early cancer based on their unique spectral signatures. Several drawbacks have limited its use for in vivo screening applications: most notably the poor temporal and spatial resolution, high expense, and low optical throughput of existing hyperspectral imagers. We present the development of a new real-time hyperspectral endoscope (called the image mapping spectroscopy endoscope) based on an image mapping technique capable of addressing these challenges. The parallel high throughput nature of this technique enables the device to operate at frame rates of 5.2 frames per second while collecting a (x, y, λ) datacube of 350 × 350 × 48. We have successfully imaged tissue in vivo, resolving a vasculature pattern of the lower lip while simultaneously detecting oxy-hemoglobin.

Multiframe image estimation for coded aperture snapshot spectral imagers

A coded aperture snapshot spectral imager (CASSI) estimates the three-dimensional spatiospectral data cube from a snapshot two-dimensional coded projection, assuming that the scene is spatially and spectrally sparse. For less spectrally sparse scenes, we show that the use of multiple nondegenerate snapshots can make data cube recovery less ill-posed, yielding improved spatial and spectral reconstruction fidelity. Additionally, data acquisition can be easily scaled to meet the time/resolution requirements of the scene with little modification or extension of the original CASSI hardware. A multiframe reconstruction of a 640 × 480 × 53 voxel datacube with 450-650 nm white-light illumination of a scene reveals substantial improvement in the reconstruction fidelity, with limited increase in acquisition and reconstruction time.

Snapshot Image Mapping Spectrometer (IMS) with high sampling density for hyperspectral microscopy

A snapshot Image Mapping Spectrometer (IMS) with high sampling density is developed for hyperspectral microscopy, measuring a datacube of dimensions 285 x 285 x 60 (x, y, lambda). The spatial resolution is approximately 0.45 microm with a FOV of 100 x 100 microm(2). The measured spectrum is from 450 nm to 650 nm and is sampled by 60 spectral channels with average sampling interval approximately 3.3 nm. The channel's spectral resolution is approximately 8nm. The spectral imaging results demonstrate the potential of the IMS for real-time cellular fluorescence imaging.

Compact Image Slicing Spectrometer (ISS) for hyperspectral fluorescence microscopy

An image slicing spectrometer (ISS) for microscopy applications is presented. Its principle is based on the redirecting of image zones by specially organized thin mirrors within a custom fabricated component termed an image slicer. The demonstrated prototype can simultaneously acquire a 140 nm spectral range within its 2D field of view from a single image. The spectral resolution of the system is 5.6 nm. The FOV and spatial resolution of the ISS depend on the selected microscope objective and for the results presented is 45 x 45 microm(2) and 0.45 microm respectively. This proof-of-concept system can be easily improved in the future for higher (both spectral and spatial) resolution imaging. The system requires no scanning and minimal post data processing. In addition, the reflective nature of the image slicer and use of prisms for spectral dispersion make the system light efficient. Both of the above features are highly valuable for real time fluorescent-spectral imaging in biological and diagnostic applications.

A hyperspectral fluorescence system for 3Din vivooptical imaging

In vivo optical instruments designed for small animal imaging generally measure the integrated light intensity across a broad band of wavelengths, or make measurements at a small number of selected wavelengths, and primarily use any spectral information to characterize and remove autofluorescence. We have developed a flexible hyperspectral imaging instrument to explore the use of spectral information to determine the 3D source location for in vivo fluorescence imaging applications. We hypothesize that the spectral distribution of the emitted fluorescence signal can be used to provide additional information to 3D reconstruction algorithms being developed for optical tomography. To test this hypothesis, we have designed and built an in vivo hyperspectral imaging system, which can acquire data from 400 to 1000 nm with 3 nm spectral resolution and which is flexible enough to allow the testing of a wide range of illumination and detection geometries. It also has the capability to generate a surface contour map of the animal for input into the reconstruction process. In this paper, we present the design of the system, demonstrate the depth dependence of the spectral signal in phantoms and show the ability to reconstruct 3D source locations using the spectral data in a simple phantom. We also characterize the basic performance of the imaging system.

Hyperspectral and multispectral bioluminescence optical tomography for small animal imaging

For bioluminescence imaging studies in small animals, it is important to be able to accurately localize the three-dimensional (3D) distribution of the underlying bioluminescent source. The spectrum of light produced by the source that escapes the subject varies with the depth of the emission source because of the wavelength-dependence of the optical properties of tissue. Consequently, multispectral or hyperspectral data acquisition should help in the 3D localization of deep sources. In this paper, we describe a framework for fully 3D bioluminescence tomographic image acquisition and reconstruction that exploits spectral information. We describe regularized tomographic reconstruction techniques that use semi-infinite slab or FEM-based diffusion approximations of photon transport through turbid media. Singular value decomposition analysis was used for data dimensionality reduction and to illustrate the advantage of using hyperspectral rather than achromatic data. Simulation studies in an atlas-mouse geometry indicated that sub-millimeter resolution may be attainable given accurate knowledge of the optical properties of the animal. A fixed arrangement of mirrors and a single CCD camera were used for simultaneous acquisition of multispectral imaging data over most of the surface of the animal. Phantom studies conducted using this system demonstrated our ability to accurately localize deep point-like sources and show that a resolution of 1.5 to 2.2 mm for depths up to 6 mm can be achieved. We also include an in vivo study of a mouse with a brain tumour expressing firefly luciferase. Co-registration of the reconstructed 3D bioluminescent image with magnetic resonance images indicated good anatomical localization of the tumour.

Lipid bilayer vesicle fusion: intermediates captured by high-speed microfluorescence spectroscopy

The fusion of lipid bilayers can be visualized under the fluorescence microscope, but the process is very fast and requires special techniques for its study. It is reported here that vesicle fusion is susceptible to analysis by microspectrofluorometry and that for the first time, the entire fusion process has been captured. In the case of giant (>10- micro m diameter) bilayer vesicles having a high density of opposite charge, fusion proceeds through stages of adhesion, flattening, hemifusion, elimination of the intervening septum, and uptake of excess membrane to generate a spherical product very rapidly. These investigations became possible with a fluorescence microscope that was modified for recording of images simultaneously with the collection of fluorescence emission spectra from many (>100) positions along the fusion axis. Positively-charged vesicles, composed of O-ethylphosphatidylcholine and dioleoylphosphatidylcholine, were labeled with a carbocyanine fluorophore. Negatively-charged vesicles, composed of dioleoylphosphatidylglycerol and dioleoylphosphatidylcholine, were labeled with a rhodamine fluorophore that is a resonance energy transfer acceptor from the carbocyanine fluorophore. An electrophoretic chamber allowed selection of pairs of vesicles to be brought into contact and examined. Spectral changes along the axis of fusion were captured at high speed (a few ms/frame) by operating a sensitive digital camera in the virtual-chip mode, a software/hardware procedure that permits rapid readout of selected regions of interest and by pixel binning along the spectral direction. Simultaneously, color images were collected at video rates (30 frame/s). Comparison of the spectra and images revealed that vesicle fusion typically passes through a hemifusion stage and that the time from vesicle contact to fusion is <10 ms. Fluorescence spectra are well suited to rapid collection in the virtual-chip mode because spectra (in contrast to images) are accurately characterized with a relatively small number of points and interfering signals can be removed by judicious choice of barrier filters. The system should be especially well-suited to phenomena exhibiting rapid fluorescence change along an axis; under optimal conditions, it is possible to obtain sets of spectra (wavelength range of approximately 150 nm) at >100 positions along a line at rates >1000 frames/s with a spectral resolution of approximately 10 nm and spatial resolution at the limit of the light microscope ( approximately 0.2 micro m).

Multispectral imaging fluorescence microscopy for living cells

Multispectral imaging technologies have been widely used in fields of astronomy and remote sensing. Interdisciplinary approaches developed in, for example, the National Aeronautics and Space Administration (NASA, USA), the Jet Propulsion Laboratory (JPL, USA), or the Communications Research Laboratory (CRL, Japan) have extended the application areas of these technologies from planetary systems to cellular systems. Here we overview multispectral imaging systems that have been devised for microscope applications. We introduce these systems with particular interest in live cell imaging. Finally we demonstrate examples of spectral imaging of living cells using commercially available systems with no need for user engineering.

Spectral imaging fluorescence microscopy

The spectral resolution of fluorescence microscope images in living cells is achieved by using a confocal laser scanning microscope equipped with grating optics. This capability of temporal and spectral resolution is especially useful for detecting spectral changes of a fluorescent dye; for example, those associated with fluorescence resonance energy transfer (FRET). Using the spectral imaging fluorescence microscope system, it is also possible to resolve emitted signals from fluorescent dyes that have spectra largely overlapping with each other, such as fluorescein isothiocyanate (FITC) and green fluorescent protein (GFP).

Large-image-format computed tomography imaging spectrometer for fluorescence microscopy

Multispectral imaging has significantly enhanced the analysis of fixed specimens in pathology and cytogenetics. However, application of this technology to in vivo studies has been limited. This is due in part to the increased temporal resolution required to analyze changes in cellular function. Here we present a non-scanning instrument that simultaneously acquires full spectral information (460 nm to 740 nm) from every pixel within its 2-D field of view (200 ìm x 200 ìm) during a single integration time (typically, 2 seconds). The current spatial and spectral sampling intervals of the spectrometer are 0.985 ìm and 5 nm, respectively. These properties allow for the analysis of physiological responses within living biological specimens.