Optical hyperspectral imaging in microscopy and spectroscopy - a review of data acquisition

A review of the medical hyperspectral imaging systems and unmixing algorithms' in biological tissues

Hyperspectral fluorescence imaging (HFI) is a well-known technique in the medical research field and is considered a non-invasive tool for tissue diagnosis. This review article gives a brief introduction to acquisition methods, including the image preprocessing methods, feature selection and extraction methods, data classification techniques and medical image analysis along with recent relevant references. The process of fusion of unsupervised unmixing techniques with other classification methods, like the combination of support vector machine with an artificial neural network, the latest snapshot Hyperspectral imaging (HSI) and vortex analysis techniques are also outlined. Finally, the recent applications of hyperspectral images in cellular differentiation of various types of cancer are discussed.

Hyperspectral imaging in wound care: A systematic review

Multispectral and hyperspectral imaging (HSI) are emerging imaging techniques with the potential to transform the way patients with wounds are cared for, but it is not clear whether current systems are capable of delivering real-time tissue characterisation and treatment guidance. We conducted a systematic review of HSI systems that have been assessed in patients, published over the past 32 years. We analysed 140 studies, including 10 different HSI systems. Current in vivo HSI systems generate a tissue oxygenation map. Tissue oxygenation measurements may help to predict those patients at risk of wound formation or delayed healing. No safety concerns were reported in any studies. A small number of studies have demonstrated the capabilities of in vivo label-free HSI, but further work is needed to fully integrate it into the current clinical workflow for different wound aetiologies. As an emerging imaging modality for medical applications, HSI offers great potential for non-invasive disease diagnosis and guidance when treating patients with both acute and chronic wounds.

Hyperspectral Imaging of Photonic Cellulose Nanocrystal Films: Structure of Local Defects and Implications for Self-Assembly Pathways

Cellulose nanocrystals (CNCs) can spontaneously assemble into chiral nematic films capable of reflecting circularly polarized light in the visible range. As many other photonic materials obtained by bottom-up approaches, CNC films often display defects that greatly impact their visual appearance. Here, we study the optical response of defects in photonic CNC films, coupling optical microscopy with hyperspectral imaging, and we compare it to optical simulations of discontinuous cholesteric structures of increasing complexity. Cross-sectional SEM observations of the film structure guided the choice of simulation parameters and showed excellent agreement with experimental optical patterns. More importantly, it strongly suggests that the last fraction of CNCs to self-assemble, upon solvent evaporation, does not undergo the typical nucleation and growth pathway, but a spinodal decomposition, an alternative self-assembly pathway so far overlooked in cast films and that can have far-reaching consequences on choices of CNC sources and assembly conditions.

Hyperspectral imaging in wound care: A systematic review

Multispectral and hyperspectral imaging (HSI) are emerging imaging techniques with the potential to transform the way patients with wounds are cared for, but it is not clear whether current systems are capable of delivering real-time tissue characterisation and treatment guidance. We conducted a systematic review of HSI systems that have been assessed in patients, published over the past 32 years. We analysed 140 studies, including 10 different HSI systems. Current in vivo HSI systems generate a tissue oxygenation map. Tissue oxygenation measurements may help to predict those patients at risk of wound formation or delayed healing. No safety concerns were reported in any studies. A small number of studies have demonstrated the capabilities of in vivo label-free HSI, but further work is needed to fully integrate it into the current clinical workflow for different wound aetiologies. As an emerging imaging modality for medical applications, HSI offers great potential for non-invasive disease diagnosis and guidance when treating patients with both acute and chronic wounds.

Hyperspectral Imaging and the Retina: Worth the Wave?

Purpose

Hyperspectral imaging is gaining attention in the biomedical field because it generates additional spectral information to study physiological and clinical processes. Several technologies have been described; however an independent, systematic literature overview is lacking, especially in the field of ophthalmology. This investigation is the first to systematically overview scientific literature specifically regarding retinal hyperspectral imaging.

Methods

A systematic literature review was conducted, in accordance with PRISMA Statement 2009 criteria, in four bibliographic databases: Medline, Embase, Cochrane Database of Systematic Reviews, and Web of Science.

Results

Fifty-six articles were found that meet the review criteria. A range of techniques was reported: Fourier analysis, liquid crystal tunable filters, tunable laser sources, dual-slit monochromators, dispersive prisms and gratings, computed tomography, fiber optics, and Fabry-Perrot cavity filter covered complementary metal oxide semiconductor. We present a narrative synthesis and summary tables of findings of the included articles, because methodologic heterogeneity and diverse research topics prevented a meta-analysis being conducted.

Conclusions

Application in ophthalmology is still in its infancy. Most previous experiments have been performed in the field of retinal oximetry, providing valuable information in the diagnosis and monitoring of various ocular diseases. To date, none of these applications have graduated to clinical practice owing to the lack of sufficiently large validation studies.

Translational Relevance

Given the promising results that smaller studies show for hyperspectral imaging (e.g., in Alzheimer's disease), advanced research in larger validation studies is warranted to determine its true clinical potential.

Application of optical technologies in the US poultry slaughter facilities for the detection of poultry carcase condemnation

1. This article reviews the studies on optical technologies for automated poultry carcase inspection, discusses challenges and potential solutions in their real-time applications in poultry slaughter facilities. 2. Over the past few decades, extensive research has been underway to develop an optical technology-based machine vision system for automated inspection of poultry carcases and viscera. Such an automated technology will not only aid in carcase inspection to maximise food safety, but it will also support the U.S. New Poultry Inspection System's aim to foster innovation in poultry processing as well as increase line speed. 3. Many earlier studies based on visible and near-infrared spectroscopy showed promise, but could not be implemented successfully in an on-line poultry processing plant. Currently, multi- and hyper-spectral imaging-based machine vision systems have shown promising outcomes. 5. The critical hurdles for real-time application of automated imaging technology in poultry carcase inspection include high-speed processing lines, slaughter facilities environment and variation in broiler rearing practices. Therefore, further improvement in imaging and machine vision technologies based on physiochemical properties on poultry carcases, the establishment of more technology friendly inspection station, and an integrated data management for different rearing practices are essential to overcome those hurdles.

Hyperspectral imaging for diagnosis and detection of ex-vivo breast cancer

BACKGROUND AND PURPOSE

Breast cancer is one of the most widely recognized tumors. .Diagnosis made in the early stage of disease may imporve outcomes. The discovery of malignant growth utilizing noninvasive light intrusive methods in lieu of conventional excisional biopsy may assist in achieving this goal.

MATERIALS AND METHODS

The change of the optical properties of ex-vivo breast tissues provides different responses to light transmission, absorption, and particularly the reflection over the spectrum range. We offer the use of Hyperspectral imaging (HSI) with advanced image processing and pattern recognition in order to analyze HSI data for breast cancer detection. The spectral signatures were mined and evaluated in both malignant and normal tissue. K-mean clustering was designed for classifying hyperspectral data in order to evaluate and detection of cancer tissue. This method was used to detect ex-vivo breast cancer. Spatial spectral images were created to high spot the differences in the reflectance properties of malignant versus normal tissue.

RESULTS

Trials showed that the superficial spectral reflection images within 500 nm wavelength showed high variance (214.65) between cancerous and normal breast tissues. On the other hand, image within 620 nm wavelength showed low variance (0.0020).However, the superimposed of spectral region 420-620 nm was proposed as the optimum bandwidth. Finally, the proposed HS imaging system was capable to discriminate the tumor region from normal tissue of the ex-vivo breast sample with sensitivity and a specificity of 95 % and 96 %.

CONCLUSIONS

High sensitivity and specificity were achieved, which proposes potential for HSI as an edge evaluation method to enhance the surgical outcome compared to the presently available techniques in the clinics.

Hyperspectral and multispectral imaging in digital and computational pathology: a systematic review [Invited]

Hyperspectral imaging (HSI) and multispectral imaging (MSI) technologies have the potential to transform the fields of digital and computational pathology. Traditional digitized histopathological slides are imaged with RGB imaging. Utilizing HSI/MSI, spectral information across wavelengths within and beyond the visual range can complement spatial information for the creation of computer-aided diagnostic tools for both stained and unstained histological specimens. In this systematic review, we summarize the methods and uses of HSI/MSI for staining and color correction, immunohistochemistry, autofluorescence, and histopathological diagnostic research. Studies include hematology, breast cancer, head and neck cancer, skin cancer, and diseases of central nervous, gastrointestinal, and genitourinary systems. The use of HSI/MSI suggest an improvement in the detection of diseases and clinical practice compared with traditional RGB analysis, and brings new opportunities in histological analysis of samples, such as digital staining or alleviating the inter-laboratory variability of digitized samples. Nevertheless, the number of studies in this field is currently limited, and more research is needed to confirm the advantages of this technology compared to conventional imagery.

Use of Fluorescent Dyes in Endoscopy and Diagnostic Investigation

Background

The advancement of innovative endoscopic technology in terms of improving the visualization of the mucosa has been of significant benefit.

Summary

Advancements in image resolution, software processing, and optical filter technology have resulted in several techniques complemental to traditional white light endoscopy. These new techniques provide a real-time optical diagnosis as well as virtual histology of detected lesions. Optical molecular imaging permits a functional assessment within cells.

Key Message

Optical molecular imaging provides an understanding of cellular processes and permits validation of the specificity of fluorescent tracers and the possibility of quantifying the signal.

Programmable hyperspectral microscopy for high-contrast biomedical imaging in a snapshot

SIGNIFICANCE

Hyperspectral microscopy has been intensively explored in biomedical applications. However, due to its huge three-dimensional hyperspectral data cube, it typically suffers from slow data acquisition, mass data transmission and storage, and computationally expensive postprocessing.

AIM

To overcome the above limitations, a programmable hyperspectral microscopy technique was developed, which can perform hardware-based hyperspectral data postprocessing by the physical process of optical imaging in a snapshot.

APPROACH

A programmable hyperspectral microscopy system was developed to collect coded microscopic images from samples under multiplexed illumination. Principal component analysis followed by linear discriminant analysis scheme was coded into multiplexed illumination and realized by the physical process of optical imaging. The contrast enhancement was evaluated on two representative types of microscopic samples, i.e., tissue section and cell samples.

RESULTS

Compared to the microscopic images collected under white light illumination, the contrasts of coded microscopic images were significantly improved by 41% and 59% for tissue section and cell samples, respectively.

CONCLUSIONS

The proposed method can perform hyperspectral data acquisition and postprocessing simultaneously by its physical process, while preserving the most important spectral information to maximize the difference between the target and background, thus opening a new avenue for high-contrast microscopic imaging in a snapshot.

Colorimetric absorbance mapping and quantitation on paper-based analytical devices

A wide range of microfluidic paper-based analytical devices (μPADs) have been developed in the last decade. Despite this, the quality of colorimetric analysis has not substantially improved as the data is vulnerable to heterogeneous color distribution (e.g., coffee ring effects), non-uniform shapes of colored detection area, and noise from the underlying paper structure. These limitations are here addressed by a colorimetric method to quantify freely discharged dye on paper substrate, without the need for a defined channel or hydrophobic barrier. For accurate quantification, colorimetric absorbance values are calculated for each pixel based on the recorded RGB values and noise from the paper structure eliminated, to extract accurate absorbance information at the pixel level. Total analyte quantity is then calculated through the conversion of absorbance values into quantity values for each pixel followed by integration across the entire image. The resulting quantity is shown to be independent of the shape of the applied colored dye spot, with a cross, circle or rod shape all giving the same quantity information. The approach is applied to a capillary-based potassium-selective sensor, where the sample solution is loaded with the dye thioflavin T (ThT) obtained by quantitative exchange with K⁺ in a sensing capillary, which is discharged onto a bare paper substrate without any channels. The resulting dye quantity is successfully obtained by flatbed scanner and smartphone. The successful automated computation of colorimetric data on μPADs will help realize simpler paper-based assay and reaction systems that should be more applicable to addressing real world analytical problems.

Investigation of the quantification of hemoglobin and cytochrome-c-oxidase in the exposed cortex with near-infrared hyperspectral imaging: a simulation study

SIGNIFICANCE

We present a Monte Carlo (MC) computational framework that simulates near-infrared (NIR) hyperspectral imaging (HSI) aimed at assisting quantification of the in vivo hemodynamic and metabolic states of the exposed cerebral cortex in small animal experiments. This can be done by targeting the NIR spectral signatures of oxygenated (HbO2) and deoxygenated (HHb) hemoglobin for hemodynamics as well as the oxidative state of cytochrome-c-oxidase (oxCCO) for measuring tissue metabolism.

AIM

The aim of this work is to investigate the performances of HSI for this specific application as well as to assess key factors for the future design and operation of a benchtop system.

APPROACH

The MC framework, based on Mesh-based Monte Carlo (MMC), reproduces a section of the exposed cortex of a mouse from an in vivo image and replicates hyperspectral illumination and detection at multiple NIR wavelengths (up to 121).

RESULTS

The results demonstrate: (1) the fitness of the MC framework to correctly simulate hyperspectral data acquisition; (2) the capability of HSI to reconstruct spatial changes in the concentrations of HbO2, HHb, and oxCCO during a simulated hypoxic condition; (3) that eight optimally selected wavelengths between 780 and 900 nm provide minimal differences in the accuracy of the hyperspectral results, compared to the "gold standard" of 121 wavelengths; and (4) the possibility to mitigate partial pathlength effects in the reconstructed data and to enhance quantification of the hemodynamic and metabolic responses.

CONCLUSIONS

The MC framework is proved to be a flexible and useful tool for simulating HSI also for different applications and targets.

Lightweight Integrated Solution for a UAV-Borne Hyperspectral Imaging System

The rapid development of unmanned aerial vehicles (UAVs), miniature hyperspectral imagers, and relevant instruments has facilitated the transition of UAV-borne hyperspectral imaging systems from concept to reality. Given the merits and demerits of existing similar UAV hyperspectral systems, we presented a lightweight, integrated solution for hyperspectral imaging systems including a data acquisition and processing unit. A pushbroom hyperspectral imager was selected owing to its superior radiometric performance. The imager was combined with a stabilizing gimbal and global-positioning system combined with an inertial measurement unit (GPS/IMU) system to form the image acquisition system. The postprocessing software included the radiance transform, surface reflectance computation, geometric referencing, and mosaic functions. The geometric distortion of the image was further significantly decreased by a postgeometric referencing software unit; this used an improved method suitable for UAV pushbroom images and showed more robust performance when compared with current methods. Two typical experiments, one of which included the case in which the stabilizing gimbal failed to function, demonstrated the stable performance of the acquisition system and data processing system. The result shows that the relative georectification accuracy of images between the adjacent flight lines was on the order of 0.7–1.5 m and 2.7–13.1 m for cases with spatial resolutions of 5.5 cm and 32.4 cm, respectively.

Applications of hyperspectral imaging in the detection and diagnosis of solid tumors

Hyperspectral imaging (HSI) is an emerging new technology in solid tumor diagnosis and detection. It incorporates traditional imaging and spectroscopy together to obtain both spatial and spectral information from tissues simultaneously in a non-invasive manner. This imaging modality is based on the principle that different tissues inherit different spectral reflectance responses that present as unique spectral fingerprints. HSI captures those composition-specific fingerprints to identify cancerous and normal tissues. It becomes a promising tool for performing tumor diagnosis and detection from the label-free histopathological examination to real-time intraoperative assistance. This review introduces the basic principles of HSI and summarizes its methodology and recent advances in solid tumor detection. In particular, the advantages of HSI applied to solid tumors are highlighted to show its potential for clinical use.

Semi-blind sparse affine spectral unmixing of autofluorescence-contaminated micrographs

MOTIVATION

Spectral unmixing methods attempt to determine the concentrations of different fluorophores present at each pixel location in an image by analyzing a set of measured emission spectra. Unmixing algorithms have shown great promise for applications where samples contain many fluorescent labels; however, existing methods perform poorly when confronted with autofluorescence-contaminated images.

RESULTS

We propose an unmixing algorithm designed to separate fluorophores with overlapping emission spectra from contamination by autofluorescence and background fluorescence. First, we formally define a generalization of the linear mixing model, called the affine mixture model (AMM), that specifically accounts for background fluorescence. Second, we use the AMM to derive an affine nonnegative matrix factorization method for estimating fluorophore endmember spectra from reference images. Lastly, we propose a semi-blind sparse affine spectral unmixing (SSASU) algorithm that uses knowledge of the estimated endmembers to learn the autofluorescence and background fluorescence spectra on a per-image basis. When unmixing real-world spectral images contaminated by autofluorescence, SSASU greatly improved proportion indeterminacy as compared to existing methods for a given relative reconstruction error.

AVAILABILITY AND IMPLEMENTATION

The source code used for this paper was written in Julia and is available with the test data at https://github.com/brossetti/ssasu.

Application of Hyperspectral Imaging as a Nondestructive Technique for Foodborne Pathogen Detection and Characterization

Microbial food safety is a persistent and exacting global issue due to the multiplicity and complexity of foods and food production systems. Foodborne illnesses caused by foodborne bacterial pathogens frequently occur, thus endangering the safety and health of human beings. Factors such as pretreatments, that is, culturing, enrichment, amplification make the traditional routine identification and enumeration of large numbers of bacteria in a complex microbial consortium complex, expensive, and time-consuming. Therefore, the need for rapid point-of-use detection systems for foodborne bacterial pathogens with high sensitivity and specificity is crucial in food safety control. Hyperspectral imaging (HSI) as a powerful testing technology provides a rapid, nondestructive approach for pathogen detection. This article reviews some fundamental information about HSI, including instrumentation, data acquisition, image processing, and data analysis-the current application of HSI for the detection, classification, and discrimination of various foodborne pathogens. The merits and demerits of HSI for pathogen detection as well as current and future trends are discussed. Therefore, the purpose of this review is to provide a brief overview of HSI, and further lay emphasis on the emerging trend and importance of this technique for foodborne pathogen detection.

Intraoperative multispectral and hyperspectral label-free imaging: A systematic review of in vivo clinical studies

Multispectral and hyperspectral imaging (HSI) are emerging optical imaging techniques with the potential to transform the way surgery is performed but it is not clear whether current systems are capable of delivering real-time tissue characterization and surgical guidance. We conducted a systematic review of surgical in vivo label-free multispectral and HSI systems that have been assessed intraoperatively in adult patients, published over a 10-year period to May 2018. We analysed 14 studies including 8 different HSI systems. Current in-vivo HSI systems generate an intraoperative tissue oxygenation map or enable tumour detection. Intraoperative tissue oxygenation measurements may help to predict those patients at risk of postoperative complications and in-vivo intraoperative tissue characterization may be performed with high specificity and sensitivity. All systems utilized a line-scanning or wavelength-scanning method but the spectral range and number of spectral bands employed varied significantly between studies and according to the system's clinical aim. The time to acquire a hyperspectral cube dataset ranged between 5 and 30 seconds. No safety concerns were reported in any studies. A small number of studies have demonstrated the capabilities of intraoperative in-vivo label-free HSI but further work is needed to fully integrate it into the current surgical workflow.

Tensor decomposition of hyperspectral images to study autofluorescence in age-related macular degeneration

Autofluorescence is the emission of light by naturally occurring tissue components on the absorption of incident light. Autofluorescence within the eye is associated with several disorders, such as Age-related Macular Degeneration (AMD) which is a leading cause of central vision loss. Its pathogenesis is incompletely understood, but endogenous fluorophores in retinal tissue might play a role. Hyperspectral fluorescence microscopy of ex-vivo retinal tissue can be used to determine the fluorescence emission spectra of these fluorophores. Comparisons of spectra in healthy and diseased tissues can provide important insights into the pathogenesis of AMD. However, the spectrum from each pixel of the hyperspectral image is a superposition of spectra from multiple overlapping tissue components. As spectra cannot be negative, there is a need for a non-negative blind source separation model to isolate individual spectra. We propose a tensor formulation by leveraging multiple excitation wavelengths to excite the tissue sample. Arranging images from different excitation wavelengths as a tensor, a non-negative tensor decomposition can be performed to recover a provably unique low-rank model with factors representing emission and excitation spectra of these materials and corresponding abundance maps of autofluorescent substances in the tissue sample. We iteratively impute missing values common in fluorescence measurements using Expectation-Maximization and use L2 regularization to reduce ill-posedness. Further, we present a framework for performing group hypothesis testing on hyperspectral images, finding significant differences in spectra between AMD and control groups in the peripheral macula. In the absence of ground truth, i.e. molecular identification of fluorophores, we provide a rigorous validation of chosen methods on both synthetic and real images where fluorescence spectra are known. These methodologies can be applied to the study of other pathologies presenting autofluorescence that can be captured by hyperspectral imaging.

Light-guide snapshot imaging spectrometer for remote sensing applications

A fiber-based snapshot imaging spectrometer was developed with a maximum of 31853 (~188 x 170) spatial sampling and 61 spectral channels in the 450nm-750nm range. A compact, custom-fabricated fiber bundle was used to sample the object image at the input and create void spaces between rows at the output for dispersion. The bundle was built using multicore 6x6 fiber block ribbons. To avoid overlap between the cores in the direction of dispersion, we selected a subset of cores using two alternative approaches; a lenslet array and a photomask. To calibrate the >30000 spatial samples of the system, a rapid spatial calibration method was developed based on phase-shifting interferometry (PSI). System crosstalk and spectral resolution were also characterized. Preliminary hyperspectral imaging results of the Rice University campus landscape, obtained with the spectrometer, are presented to demonstrate the system's spectral imaging capability for distant scenes. The spectrum of different plant species with different health conditions, obtained with the spectrometer, was in accordance with reference instrument measurements. We also imaged Houston traffic to demonstrate the system's snapshot hyperspectral imaging capability. Potential applications of the system include terrestrial monitoring, land use, air pollution, water resources, and lightning spectroscopy. The fiber-based system design potentially allows tuning between spatial and spectral sampling to meet specific imaging requirements.

Custom Scanning Hyperspectral Imaging System for Biomedical Applications: Modeling, Benchmarking, and Specifications

Prototyping hyperspectral imaging devices in current biomedical optics research requires taking into consideration various issues regarding optics, imaging, and instrumentation. In summary, an ideal imaging system should only be limited by exposure time, but there will be technological limitations (e.g., actuator delay and backlash, network delays, or embedded CPU speed) that should be considered, modeled, and optimized. This can be achieved by constructing a multiparametric model for the imaging system in question. The article describes a rotating-mirror scanning hyperspectral imaging device, its multiparametric model, as well as design and calibration protocols used to achieve its optimal performance. The main objective of the manuscript is to describe the device and review this imaging modality, while showcasing technical caveats, models and benchmarks, in an attempt to simplify and standardize specifications, as well as to incentivize prototyping similar future designs.

Compact snapshot optically replicating and remapping imaging spectrometer (ORRIS) using a focal plane continuous variable filter

In this Letter, a novel snapshot spectral imaging technique, optically replicating and remapping imaging spectrometer, is presented. It is based on the combination of shifting subimages by a specially designed lenslet array (LA) and filtering subimages by a focal plane continuous variable filter (CVF). The 3D datacube is recovered by just using a simple image remapping process. The use of the LA and the focal plane CVF makes the system compact and low in cost. A handheld proof-of-principle prototype has been built and demonstrated; it covers a wavelength range of 380-860 nm with 80 spectral channels with a spatial resolution of 400×400  pixels.

Multicolor fluorescent imaging by space-constrained computational spectral imaging

Spectral imaging is a powerful technique used to simultaneously study multiple fluorophore labels with overlapping emissions. Here, we present a computational spectral imaging method, which uses sample spatial fluorescence information as a reconstruction constraint. Our method addresses both the under-sampling issue of compressive spectral imaging and the low throughput issue of scanning spectral imaging. With simulated and experimental data, we have demonstrated the reconstruction precision of our method in two and three-color imaging. We have experimentally validated this method for differentiating cellular structures labeled with two red-colored fluorescent proteins, tdTomato and mCherry, which have highly overlapping emission spectra. Our method has the advantage of totally free wavelength choice and can also be combined with conventional filter-based sequential multi-color imaging to further improve multiplexing capability.

Hyperspectral imaging analysis of a photonic crystal bead array for multiplex bioassays

For multiplex bioassays, one effective strategy is to employ microfluidic chips based on an array of photonic crystal beads (PCBs) that are encoded by their characteristic reflection spectrum (CRS). In this paper, we report a hyperspectral imaging system and algorithms for the high throughput decoding of a PCB array and subsequent detection. The results showed that the decoding accuracy of up to ∼500 PCBs is 98.56% with an excellent ability to extract low-intensity fluorescence intensities. The results also demonstrated hyperspectral imaging techniques which can simultaneously obtain both spatial and spectral information as powerful tools in the analysis of multiplex bioassays or microfluidic chips.

Adverse Interactions of Luminescent Semiconductor Quantum Dots with Liposomes and Shewanella oneidensis

Cadmium-containing luminescent quantum dots (QD) are increasingly used in display, bioimaging, and energy technologies; however, significant concerns have been raised about their potentially adverse impact on human health and the environment. This study makes use of a broad toolkit of analytical methods to investigate and increase our understanding of the interactions of luminescent cadmium-containing (CdSe) and cadmium-free (ZnSe) QD, with and without a passivating higher bandgap energy ZnS shell, with phospholipid vesicles (liposomes), which model bacterial membranes, and with Shewanella oneidensis MR-1, an environmentally relevant bacteria. A unique feature of this study is that all QD types have the same surface chemistry, being capped with uncharged poly(ethylene glycol) ligands. This enables focusing the study on the impact of the QD core on liposomes and bacterial cells. The study reveals that QD association with liposome and bacterial cell membranes is imperative for their adverse impact on liposomes and bacterial cells. The QD' concentration-dependent association with liposomes and bacterial cells destabilizes the membranes mechanically, which leads to membrane disruption and lysis in liposomes and to bacterial cell death. The study also shows that cadmium-containing QD exhibit a higher level of membrane disruption in bacterial cells than cadmium-free QD. ZnSe QD have low membrane impact, and coating them with a ZnS shell decreases their membrane disruption activity. In contrast, CdSe QD exhibit a high level of membrane impact, and coating them with a ZnS shell does not decrease, but in fact further increases, their membrane disruption activity. This behavior might be attributed to higher affinity and association of CdSe/ZnS QD with liposomes and bacterial cells and to a contribution of dissolved zinc ions from the ZnS shell to increased membrane disruption activity.

Fast hyperspectral phase and amplitude imaging in scattering tissue

Hyperspectral imaging in scattering tissue generally suffers from low light collection efficiency. In this Letter, we propose a microscope based on Fourier transform spectroscopy and oblique back-illumination microscopy that provides hyperspectral phase and amplitude images of thick, scattering samples with high throughput. Images can be acquired at >0.1  Hz rates with spectral resolution better than 200  cm-1, over a wide spectral range of 450-1700 nm. Proof-of-principle demonstrations are presented with chorioallantoic membrane of a chick embryo, illustrating the possibility of high-resolution hemodynamics imaging in thick tissue, based on transmission contrast.

Multispectral analysis tools can increase utility of RGB color images in histology

Multispectral imaging (MSI) is increasingly finding application in the study and characterization of biological specimens. However, the methods typically used come with challenges on both the acquisition and the analysis front. MSI can be slow and photon-inefficient, leading to long imaging times and possible phototoxicity and photobleaching. The resulting datasets can be large and complex, prompting the development of a number of mathematical approaches for segmentation and signal unmixing. We show that under certain circumstances, just three spectral channels provided by standard color cameras, coupled with multispectral analysis tools, including a more recent spectral phasor approach, can efficiently provide useful insights. These findings are supported with a mathematical model relating spectral bandwidth and spectral channel number to achievable spectral accuracy. The utility of 3-band RGB and MSI analysis tools are demonstrated on images acquired using brightfield and fluorescence techniques, as well as a novel microscopy approach employing UV-surface excitation. Supervised linear unmixing, automated non-negative matrix factorization and phasor analysis tools all provide useful results, with phasors generating particularly helpful spectral display plots for sample exploration.

Hyperspectral imaging solutions for brain tissue metabolic and hemodynamic monitoring: past, current and future developments

Hyperspectral imaging (HSI) technologies have been used extensively in medical research, targeting various biological phenomena and multiple tissue types. Their high spectral resolution over a wide range of wavelengths enables acquisition of spatial information corresponding to different light-interacting biological compounds. This review focuses on the application of HSI to monitor brain tissue metabolism and hemodynamics in life sciences. Different approaches involving HSI have been investigated to assess and quantify cerebral activity, mainly focusing on: (1) mapping tissue oxygen delivery through measurement of changes in oxygenated (HbO2) and deoxygenated (HHb) hemoglobin; and (2) the assessment of the cerebral metabolic rate of oxygen (CMRO2) to estimate oxygen consumption by brain tissue. Finally, we introduce future perspectives of HSI of brain metabolism, including its potential use for imaging optical signals from molecules directly involved in cellular energy production. HSI solutions can provide remarkable insight in understanding cerebral tissue metabolism and oxygenation, aiding investigation on brain tissue physiological processes.

Hyperspectral-Enhanced Dark Field Microscopy for Single and Collective Nanoparticle Characterization in Biological Environments

We review how the hyperspectral dark field analysis gives us quantitative insights into the manner that different nanoscale materials interact with their environment and how this relationship is directly expressed in an optical readout. We engage classification tools to identify dominant spectral signatures within a scene or to qualitatively characterize nanoparticles individually or in populations based on their composition and morphology. Moreover, we follow up the morphological evolution of nanoparticles over time and in different biological environments to better understand and establish a link between the observed nanoparticles' changes and cellular behaviors.

Macroscale fluorescence imaging against autofluorescence under ambient light

Macroscale fluorescence imaging is increasingly used to observe biological samples. However, it may suffer from spectral interferences that originate from ambient light or autofluorescence of the sample or its support. In this manuscript, we built a simple and inexpensive fluorescence macroscope, which has been used to evaluate the performance of Speed OPIOM (Out of Phase Imaging after Optical Modulation), which is a reference-free dynamic contrast protocol, to selectively image reversibly photoswitchable fluorophores as labels against detrimental autofluorescence and ambient light. By tuning the intensity and radial frequency of the modulated illumination to the Speed OPIOM resonance and adopting a phase-sensitive detection scheme that ensures noise rejection, we enhanced the sensitivity and the signal-to-noise ratio for fluorescence detection in blot assays by factors of 50 and 10, respectively, over direct fluorescence observation under constant illumination. Then, we overcame the strong autofluorescence of growth media that are currently used in microbiology and realized multiplexed fluorescence observation of colonies of spectrally similar fluorescent bacteria with a unique configuration of excitation and emission wavelengths. Finally, we easily discriminated fluorescent labels from the autofluorescent and reflective background in labeled leaves, even under the interference of incident light at intensities that are comparable to sunlight. The proposed approach is expected to find multiple applications, from biological assays to outdoor observations, in fluorescence macroimaging.

Multiplexed Optical Imaging of Tumor-Directed Nanoparticles: A Review of Imaging Systems and Approaches

In recent decades, various classes of nanoparticles have been developed for optical imaging of cancers. Many of these nanoparticles are designed to specifically target tumor sites, and specific cancer biomarkers, to facilitate the visualization of tumors. However, one challenge for accurate detection of tumors is that the molecular profiles of most cancers vary greatly between patients as well as spatially and temporally within a single tumor mass. To overcome this challenge, certain nanoparticles and imaging systems have been developed to enable multiplexed imaging of large panels of cancer biomarkers. Multiplexed molecular imaging can potentially enable sensitive tumor detection, precise delineation of tumors during interventional procedures, and the prediction/monitoring of therapy response. In this review, we summarize recent advances in systems that have been developed for the imaging of optical nanoparticles that can be heavily multiplexed, which include surface-enhanced Raman-scattering nanoparticles (SERS NPs) and quantum dots (QDs). In addition to surveying the optical properties of these various types of nanoparticles, and the most-popular multiplexed imaging approaches that have been employed, representative preclinical and clinical imaging studies are also highlighted.

Fluorescence hyperspectral imaging (fHSI) using a spectrally resolved detector array

The ability to resolve multiple fluorescent emissions from different biological targets in video rate applications, such as endoscopy and intraoperative imaging, has traditionally been limited by the use of filter-based imaging systems. Hyperspectral imaging (HSI) facilitates the detection of both spatial and spectral information in a single data acquisition, however, instrumentation for HSI is typically complex, bulky and expensive. We sought to overcome these limitations using a novel robust and low cost HSI camera based on a spectrally resolved detector array (SRDA). We integrated this HSI camera into a wide-field reflectance-based imaging system operating in the near-infrared range to assess the suitability for in vivo imaging of exogenous fluorescent contrast agents. Using this fluorescence HSI (fHSI) system, we were able to accurately resolve the presence and concentration of at least 7 fluorescent dyes in solution. We also demonstrate high spectral unmixing precision, signal linearity with dye concentration and at depth in tissue mimicking phantoms, and delineate 4 fluorescent dyes in vivo. Our approach, including statistical background removal, could be directly generalised to broader spectral ranges, for example, to resolve tissue reflectance or autofluorescence and in future be tailored to video rate applications requiring snapshot HSI data acquisition.

Ultraminiature optical design for multispectral fluorescence imaging endoscopes

A miniature wide-field multispectral endoscopic imaging system was developed enabling reflectance and fluorescence imaging over a broad wavelength range. At 0.8-mm diameter, the endoscope can be utilized for natural orifice imaging in small lumens such as the fallopian tubes. Five lasers from 250 to 642 nm are coupled into a 125 - ? m diameter multimode fiber and transmitted to the endoscope distal tip for illumination. Ultraviolet and blue wavelengths excite endogenous fluorophores, which can provide differential fluorescence emission images for health and disease. Visible wavelengths provide reflectance images that can be combined for pseudo-white-light imaging and navigation. Imaging is performed by a 300 - ? m diameter three-element lens system connected to a 3000-element fiber. The lens system was designed for a 70-deg full field of view, working distance from 3 mm to infinity, and 40% contrast at the Nyquist cutoff of the fiber bundle. Measured performance characteristics are near design goals. The endoscope was utilized to obtain example monochromatic, pseudo-white-light, and composite fluorescence images of phantoms and porcine reproductive tract. This work shows the feasibility of packaging a highly capable multispectral fluorescence imaging system into a miniature endoscopic system that may have applications in early detection of cancer.

High spatial sampling light-guide snapshot spectrometer

A prototype fiber-based imaging spectrometer was developed to provide snapshot hyperspectral imaging tuned for biomedical applications. The system is designed for imaging in the visible spectral range from 400 to 700 nm for compatibility with molecular imaging applications as well as satellite and remote sensing. An 81 × 96 pixel spatial sampling density is achieved by using a custom-made fiber-optic bundle. The design considerations and fabrication aspects of the fiber bundle and imaging spectrometer are described in detail. Through the custom fiber bundle, the image of a scene of interest is collected and divided into discrete spatial groups, with spaces generated in between groups for spectral dispersion. This reorganized image is scaled down by an image taper for compatibility with following optical elements, dispersed by a prism, and is finally acquired by a CCD camera. To obtain an (x, y, λ) datacube from the snapshot measurement, a spectral calibration algorithm is executed for reconstruction of the spatial-spectral signatures of the observed scene. System characterization of throughput, resolution, and crosstalk was performed. Preliminary results illustrating changes in oxygen-saturation in an occluded human finger are presented to demonstrate the system's capabilities.

Phase and Index of Refraction Imaging by Hyperspectral Reflectance Confocal Microscopy

A hyperspectral reflectance confocal microscope (HSCM) was realized by CNR-ISC (Consiglio Nazionale delle Ricerche-Istituto dei Sistemi Complessi) a few years ago. The instrument and data have been already presented and discussed. The main activity of this HSCM has been within biology, and reflectance data have shown good matching between spectral signatures and the nature or evolution on many types of cells. Such a relationship has been demonstrated mainly with statistical tools like Principal Component Analysis (PCA), or similar concepts, which represent a very common approach for hyperspectral imaging. However, the point is that reflectance data contains much more useful information and, moreover, there is an obvious interest to go from reflectance, bound to the single experiment, to reflectivity, or other physical quantities, related to the sample alone. To accomplish this aim, we can follow well-established analyses and methods used in reflectance spectroscopy. Therefore, we show methods of calculations for index of refraction n, extinction coefficient k and local thicknesses of frequency starting from phase images by fast Kramers-Kronig (KK) algorithms and the Abeles matrix formalism. Details, limitations and problems of the presented calculations as well as alternative procedures are given for an example of HSCM images of red blood cells (RBC).

Segmentation in dermatological hyperspectral images: dedicated methods

Background

Segmentation of hyperspectral medical images is one of many image segmentation methods which require profiling. This profiling involves either the adjustment of existing, known image segmentation methods or a proposal of new dedicated methods of hyperspectral image segmentation. Taking into consideration the size of analysed data, the time of analysis is of major importance. Therefore, the authors proposed three new dedicated methods of hyperspectral image segmentation with special reference to the time of analysis.

Methods

The segmentation methods presented in this paper were tested and profiled to the images acquired from different hyperspectral cameras including SOC710 Hyperspectral Imaging System, Specim sCMOS-50-V10E. Correct functioning of the method was tested for over 10,000 2D images constituting the sequence of over 700 registrations of the areas of the left and right hand and the forearm.

Results

As a result, three new methods of hyperspectral image segmentation have been proposed: fast analysis of emissivity curves (SKE), 3D segmentation (S3D) and hierarchical segmentation (SH). They have the following features: are fully automatic; allow for implementation of fast segmentation methods; are profiled to hyperspectral image segmentation; use emissivity curves in the model form, can be applied in any type of objects not necessarily biological ones, are faster (SKE—2.3 ms, S3D—1949 ms, SH—844 ms for the computer with Intel^® Core i7 4960X CPU 3.6 GHz) and more accurate (SKE—accuracy 79 %, S3D—90 %, SH—92 %) in comparison with typical methods known from the literature.

Conclusions

Profiling and/or proposing new methods of hyperspectral image segmentation is an indispensable element of developing software. This ensures speed, repeatability and low sensitivity of the algorithm to changing parameters.

The use of hyperspectral imaging in the VNIR (400-1000nm) and SWIR range (1000-2500nm) for detecting counterfeit drugs with identical API composition

The risk of death from taking counterfeit drugs is now greater than the probability of dying from malaria and AIDS combined (at least half a million deaths each year). At the same time, counterfeit medicines are falsified more and more "skillfully". According to WHO about 10% of counterfeit drugs are copies of original products. The methods of hyperspectral imaging and image analysis and processing were used to detect counterfeit drugs. Original Viagra® (Pfizer) and counterfeit tablets were compared. Hyperspectral imaging was used to acquire hyperspectral data cubes from both original and counterfeit tablets in the spectral range of 400-2500nm. Spectral parameters for both the original Viagra® and counterfeit drugs were compared. Grey-Level Co-Occurrence Matrix (GLCM) analysis and Principal Component Analysis (PCA) were performed. Hyperspectral analysis of the surface of the original Viagra® and counterfeit tablets demonstrates significant differences in reflectance (maximum difference for 1619.75nm). The GLCM contrast for the falsified drug is on average higher than for the original one 16±4%. GLCM contrast analysis enables to quantify homogeneity of distribution of tablet ingredients and enables to distinguish tablets with identical chemical composition. SWIR (1000-2500nm) hyperspectral imaging has a definite advantage over imaging in VNIR (400-1000nm) - higher wavelength is less sensitive to non-uniform illumination.

Label-free and non-invasive discrimination of HaCaT and melanoma cells in a co-culture model by hyperspectral confocal reflectance microscopy

A novel hyperspectral confocal microscopy method to separate different cell populations in a co-culture model is presented here. The described methodological and instrumental approach allows discrimination of different cell types using a non-invasive, label free method with good accuracy with a single cell resolution. In particular, melanoma cells are discriminated from HaCaT cells by hyperspectral confocal imaging, principal component analysis and optical frequencies signing, as confirmed by fluorescence labelling cross check. The identification seems to be quite robust to be insensitive to the cellular shape within the studied samples, enabling to separate cells according to their cytotype down to a single cell sensitivity. Set of hyperspectral images of melanoma-keratinocytes co-culture model (left), score plot of principal component analysis and spectral analysis of principal components coefficients (center), label-free spectral identification of cell populations (right).

Spatial and Spectral Characterization of Human Retinal Pigment Epithelium Fluorophore Families by Ex Vivo Hyperspectral Autofluorescence Imaging

Purpose

Discovery of candidate spectra for abundant fluorophore families in human retinal pigment epithelium (RPE) by ex vivo hyperspectral imaging.

Methods

Hyperspectral autofluorescence emission images were captured between 420 and 720 nm (10-nm intervals), at two excitation bands (436–460, 480–510 nm), from three locations (fovea, perifovea, near-periphery) in 20 normal RPE/Bruch's membrane (BrM) flatmounts. Mathematical factorization extracted a BrM spectrum (S0) and abundant lipofuscin/melanolipofuscin (LF/ML) spectra of RPE origin (S1, S2, S3) from each tissue.

Results

Smooth spectra S1 to S3, with perinuclear localization consistent with LF/ML at all three retinal locations and both excitations in 14 eyes (84 datasets), were included in the analysis. The mean peak emissions of S0, S1, and S2 at λ_ex 436 nm were, respectively, 495 ± 14, 535 ± 17, and 576 ± 20 nm. S3 was generally trimodal, with peaks at either 580, 620, or 650 nm (peak mode, 650 nm). At λ_ex 480 nm, S0, S1, and S2 were red-shifted to 526 ± 9, 553 ± 10, and 588 ± 23 nm, and S3 was again trimodal (peak mode, 620 nm). S1 often split into two spectra, S1A and S1B. S3 strongly colocalized with melanin. There were no significant differences across age, sex, or retinal location.

Conclusions

There appear to be at least three families of abundant RPE fluorophores that are ubiquitous across age, retinal location, and sex in this sample of healthy eyes. Further molecular characterization by imaging mass spectrometry and localization via super-resolution microscopy should elucidate normal and abnormal RPE physiology involving fluorophores.

Translational Relevance

Our results help establish hyperspectral autofluorescence imaging of the human retinal pigment epithelium as a useful tool for investigating retinal health and disease.

Hyperspectral multiplex single-particle tracking of different receptor subtypes labeled with quantum dots in live neurons

The efficacy of existing therapies and the discovery of innovative treatments for central nervous system (CNS) diseases have been limited by the lack of appropriate methods to investigate complex molecular processes at the synaptic level. To improve our capability to investigate complex mechanisms of synaptic signaling and remodeling, we designed a fluorescence hyperspectral imaging platform to simultaneously track different subtypes of individual neurotransmitter receptors trafficking in and out of synapses. This imaging platform allows simultaneous image acquisition of at least five fluorescent markers in living neurons with a high-spatial resolution. We used quantum dots emitting at different wavelengths and functionalized to specifically bind to single receptors on the membrane of living neurons. The hyperspectral imaging platform enabled the simultaneous optical tracking of five different synaptic proteins, including subtypes of glutamate receptors (mGluR and AMPAR) and postsynaptic signaling proteins. It also permitted the quantification of their mobility after treatments with various pharmacological agents. This technique provides an efficient method to monitor several synaptic proteins at the same time, which could accelerate the screening of effective compounds for treatment of CNS disorders.

A review of snapshot multidimensional optical imaging: measuring photon tags in parallel

Multidimensional optical imaging has seen remarkable growth in the past decade. Rather than measuring only the two-dimensional spatial distribution of light, as in conventional photography, multidimensional optical imaging captures light in up to nine dimensions, providing unprecedented information about incident photons' spatial coordinates, emittance angles, wavelength, time, and polarization. Multidimensional optical imaging can be accomplished either by scanning or parallel acquisition. Compared with scanning-based imagers, parallel acquisition-also dubbed snapshot imaging-has a prominent advantage in maximizing optical throughput, particularly when measuring a datacube of high dimensions. Here, we first categorize snapshot multidimensional imagers based on their acquisition and image reconstruction strategies, then highlight the snapshot advantage in the context of optical throughput, and finally we discuss their state-of-the-art implementations and applications.