Multi-mode Microscopic Hyperspectral Imager for the Sensing of Biological Samples

Nondestructive detection of saponin content in Panax notoginseng powder based on hyperspectral imaging

This study investigated the feasibility of using hyperspectral imaging (HSI) technique to detect the saponin content in Panax notoginseng (PN) powder. The reflectance hyperspectral images of PN powder samples were collected in the spectral range of 400.6-999.9 nm. Savitzky-golay (SG) smoothing combined with detrending correction was utilized to preprocess the original spectral data. Two model population analysis (MPA) based methods, namely bootstrapping soft shrinkage (BOSS) and iteratively retains informative variables (IRIV) were employed to extract feature wavelengths from the full spectra. A generalized normal distribution optimization based extreme learning machine (GNDO-ELM) model was proposed to establish calibration model between spectra and saponin content, and compared with existing methods (GA-ELM, PSO-ELM and SSA-ELM). The result showed that the IRIV-GNDO-ELM model gave the best performance, with coefficient of determination for prediction (R2P) of 0.953 and root mean square error for prediction (RMSEP) of 0.115%. Therefore, it is possible to determine the saponin content of PN powder by using HSI technique.

Polarized Hyperspectral Microscopic Imaging for Zebrafish

Zebrafish is a well-established animal model for developmental and disease studies. Its optical transparency at early developmental stages is ideal for tissue visualization. Interaction of light with zebrafish tissues provides information on their structure and properties. In this study, we developed a microscopic imaging system for improving the visualization of unstained zebrafish tissues on tissue slides, with two different setups: polarized light imaging and polarized hyperspectral imaging. Based on the polarized light imaging setup, we collected the RGB images of Stokes vector parameters (S0, S1, S2, and S3), and calculated the Stokes vector derived parameters: the degree of polarization (DOP), the degree of linear polarization (DOLP)). We also calculated Stokes vector data based on the polarized hyperspectral imaging setup. The preliminary results demonstrate that Stokes vector data in two imaging setups (polarized light imaging and polarized hyperspectral imaging) are capable of improving the visualization of different types of zebrafish tissues (brain, muscle, skin cells, blood vessels, and yolk). Using the images collected from larval zebrafish samples by polarized light imaging, we found that DOP and DOLP could show clearer structural information of the brain and of skin cells, muscle and blood vessels in the tail. Furthermore, DOP and DOLP parameters derived from images collected by polarized hyperspectral imaging could show clearer structural information of skin cells developing around yolk as well as the surrounding blood vessel network. In addition, polarized hyperspectral imaging could provide complementary spectral information to the spatial information on Stokes vector data of zebrafish tissues. The polarized light imaging & polarized hyperspectral imaging systems provide a better insight into the microstructures of zebrafish tissues.

Assessment of hyperspectral imaging and CycleGAN-simulated narrowband techniques to detect early esophageal cancer

The clinical signs and symptoms of esophageal cancer (EC) are often not discernible until the intermediate or advanced phases. The detection of EC in advanced stages significantly decreases the survival rate to below 20%. This study conducts a comparative analysis of the efficacy of several imaging techniques, including white light image (WLI), narrowband imaging (NBI), cycle-consistent adversarial network simulated narrowband image (CNBI), and hyperspectral imaging simulated narrowband image (HNBI), in the early detection of esophageal cancer (EC). In conjunction with Kaohsiung Armed Forces General Hospital, a dataset consisting of 1000 EC pictures was used, including 500 images captured using WLI and 500 images captured using NBI. The CycleGAN model was used to generate the CNBI dataset. Additionally, a novel method for HSI imaging was created with the objective of generating HNBI pictures. The evaluation of the efficacy of these four picture types in early detection of EC was conducted using three indicators: CIEDE2000, entropy, and the structural similarity index measure (SSIM). Results of the CIEDE2000, entropy, and SSIM analyses suggest that using CycleGAN to generate CNBI images and HSI model for creating HNBI images is superior in detecting early esophageal cancer compared to the use of conventional WLI and NBI techniques.

4D hyperspectral surface topography measurement system based on the Scheimpflug principle and hyperspectral imaging

A four-dimensional (4D) hyperspectral surface topography measurement (HSTM) system that can acquire uniform inelastic signals [three-dimensional (3D) spatial data] and reflection/fluorescence spectra of an object is proposed. The key components of the system are a light-sheet profilometer based on the Scheimpflug principle and a hyperspectral imager. Based on the mapping relationships among the image coordinate systems of the two imaging subsystems and the coordinate system of the real space, the spectral data can be assigned to the corresponding 3D point cloud, forming a 4D model. The spectral resolution is better than 4 nm. 700 nm, 546 nm, and 436 nm are selected as the three primary colors of red, green, and blue to restore the color. The 4D hyperspectral surface reconstruction experiments of philodendron and chlorophytum have shown the good performance of the proposed HSTM system and the great application potential for plant phenotype and growth analysis in agriculture.

Spatially encoded hyperspectral compressive microscope for ultrabroadband VIS/NIR hyperspectral imaging

Hyperspectral imaging (HSI) has become a valuable tool in sample characterization in various scientific fields. While many approaches have been tested, specific applications and technology usually lead to only a narrow part of the spectrum being studied. We demonstrate the use of a broadband HSI setup based on compressed sensing capable of capturing data in visible (VIS), near-infrared (NIR), and short-wave infrared (SWIR) spectral regions. Using a tested design, we developed a dual configuration and tested its performance on a set of samples demonstrating spatial resolution and spectral reconstruction. Samples showing a potential use of the setup in optical defect detection are also tested. The setup showcases a dual single-pixel camera configuration capable of combining various detectors with a shared spatial modulation, further improving data efficiency and providing an affordable instrument from broadband spectral studies.

Compact and ultracompact spectral imagers: technology and applications in biomedical imaging

Significance

Spectral imaging, which includes hyperspectral and multispectral imaging, can provide images in numerous wavelength bands within and beyond the visible light spectrum. Emerging technologies that enable compact, portable spectral imaging cameras can facilitate new applications in biomedical imaging.

Aim

With this review paper, researchers will (1) understand the technological trends of upcoming spectral cameras, (2) understand new specific applications that portable spectral imaging unlocked, and (3) evaluate proper spectral imaging systems for their specific applications.

Approach

We performed a comprehensive literature review in three databases (Scopus, PubMed, and Web of Science). We included only fully realized systems with definable dimensions. To best accommodate many different definitions of "compact," we included a table of dimensions and weights for systems that met our definition.

Results

There is a wide variety of contributions from industry, academic, and hobbyist spaces. A variety of new engineering approaches, such as Fabry-Perot interferometers, spectrally resolved detector array (mosaic array), microelectro-mechanical systems, 3D printing, light-emitting diodes, and smartphones, were used in the construction of compact spectral imaging cameras. In bioimaging applications, these compact devices were used for in vivo and ex vivo diagnosis and surgical settings.

Conclusions

Compact and ultracompact spectral imagers are the future of spectral imaging systems. Researchers in the bioimaging fields are building systems that are low-cost, fast in acquisition time, and mobile enough to be handheld.

Determination of geographic origins and types of Lindera aggregata samples using a portable short-wave infrared hyperspectral imager

A portable short-wavelength infrared microscope hyperspectral imager (SMHI) combined with machine learning algorithms for the purpose of classifying geographical origins as well as root types of Lindera aggregata is developed. The spectral range of the SMHI system is 1090-1820 nm (5500-9100 cm-1) with spectral and spatial resolutions of 4 nm and 27.3 μm, respectively. Utilizing PCA-RF algorithms, the geographic origin of tuberous roots and leaves from five different origins were classified with accuracies of 97.5% and 97.8%, respectively. In addition, spatial identification of tuberous root and taproot tubers in a mixed sample was done with an accuracy of 98.98%. The accuracy of origin classification and spatial identification are high enough which indicate the significant potential of applying SMHI system into the non-invasive spatial mapping and rapid quality assessment of medicinal herbs.

Wavelength selection of multispectral imaging for oil palm fresh fruit ripeness classification

Multispectral imaging has been recently proposed for high-speed sorting and grading machine vision of fruits. It is a prospective method applied in yet traditional sorting and grading of oil palm fresh fruit bunches (FFB). The ripeness of oil palm FFBs determines the quality of crude palm oil (CPO). Implementation of multispectral imaging for the task needs wavelength selection from hyperspectral datasets. This study aimed to obtain the optimum wavelengths and use them for oil palm FFB classification based on three ripeness levels. We have selected eight optimum wavelengths using principal component analysis (PCA) regression which represented the ripeness levels.

Versatile compressive microscope for hyperspectral transmission and fluorescence lifetime imaging

Increasing demand for multimodal characterization and imaging of new materials entails the combination of various methods in a single microscopic setup. Hyperspectral imaging of transmission spectra or photoluminescence (PL) decay imaging count among the most used methods. Nevertheless, these methods require very different working conditions and instrumentation. Therefore, combining the methods into a single microscopic system is seldom implemented. Here we demonstrate a novel versatile microscope based on single-pixel imaging, where we use a simple optical configuration to measure the hyperspectral information, as well as fluorescence lifetime imaging (FLIM). The maps are inherently spatially matched and can be taken with spectral resolution limited by the resolution of the used spectrometer (3 nm) or temporal resolution set by PL decay measurement (120 ps). We verify the system's performance by its comparison to the standard FLIM and non-imaging transmission spectroscopy. Our approach enabled us to switch between a broad field-of-view and micrometer resolution without changing the optical configuration. At the same time, the used design opens the possibility to add a variety of other characterization methods. This article demonstrates a simple, affordable way of complex material studies with huge versatility for the imaging parameters.

Incoherent broadband cavity-enhanced absorption spectroscopy for sensitive measurement of nutrients and microalgae

Incoherent broadband cavity-enhanced absorption spectroscopy (IBBCEAS) can achieve sensitive measurements at trace concentrations for liquid phase marine samples. The IBBCEAS system consists of a cavity-enhancement module (CEM) and a transmission hyperspectral module (THM). The CEM has cavity-enhancement factors up to 78 at 550 nm. Measurements were obtained over a wide wavelength range (420-640 nm) with a halogen lamp, and the optical cavity was formed by two concave highly reflective mirrors (R=0.99). The minimum detectable absorption coefficient α_min of 7.3×10^-7cm^-1 at 550 nm corresponds to a limit of detection for nutrients of 780 pM. The spectral resolution of the THM is 3 nm in the wavelength range of 400 to 750 nm. We performed the IBBCEAS measurements for biological and chemical substances, including nutrients, microalgae, and Cy5 dye. The concentrations of nutrients in a deionized water environment and artificial seawater environment were measured at nanomolar levels; the concentration of microalgae phaeocystis was detected with 3.46×10⁴/mL, and fluorescence substances such as Cy5 dye could be measured at 0.03 mg/L. Experimental results show that the IBBCEAS system has the capability for sensitive measurements of biological and chemical substances and has strong potential forin situ ecological marine environmental monitoring function.

Machine learning classification of origins and varieties of Tetrastigma hemsleyanum using a dual-mode microscopic hyperspectral imager

A dual-mode microscopic hyperspectral imager (DMHI) combined with a machine learning algorithm for the purpose of classifying origins and varieties of Tetrastigma hemsleyanum (T. hemsleyanum) was developed. By switching the illumination source, the DMHI can operate in reflection imaging and fluorescence detection modes. The DMHI system has excellent performance with spatial and spectral resolutions of 27.8 μm and 3 nm, respectively. To verify the capability of the DMHI system, a series of classification experiments of T. hemsleyanum were conducted. Captured hyperspectral datasets were analyzed using principal component analysis (PCA) for dimensional reduction, and a support vector machine (SVM) model was used for classification. In reflection microscopic hyperspectral imaging (RMHI) mode, the classification accuracies of T. hemsleyanum origins and varieties were 96.3% and 97.3%, respectively, while in fluorescence microscopic hyperspectral imaging (FMHI) mode, the classification accuracies were 97.3% and 100%, respectively. Combining datasets in dual mode, excellent predictions of origin and variety were realized by the trained model, both with a 97.5% accuracy on a newly measured test set. The results show that the DMHI system is capable of T. hemsleyanum origin and variety classification, and has the potential for non-invasive detection and rapid quality assessment of various kinds of medicinal herbs.

Water monitoring by means of digital microscopy identification and classification of microalgae

Marine and freshwater microalgae belong to taxonomically and morphologically diverse groups of organisms spanning many phyla with thousands of species. These organisms play an important role as indicators of water ecosystem conditions since they react quickly and predictably to a broad range of environmental stressors, thus providing early signals of dangerous changes. Traditionally, microscopic analysis has been used to identify and enumerate different types of organisms present within a given environment at a given point in time. However, this approach is both time-consuming and labor intensive, as it relies on manual processing and classification of planktonic organisms present within collected water samples. Furthermore, it requires highly skilled specialists trained to recognize and distinguish one taxa from another on the basis of often subtle morphological differences. Given these restrictions, a considerable amount of effort has been recently funneled into automating different steps of both the sampling and classification processes, making it possible to generate previously unprecedented volumes of plankton image data and obtain an essential database to analyze the composition of plankton assemblages. In this review we report state-of-the-art methods used for automated plankton classification by means of digital microscopy. The computer-microscope system hardware and the image processing techniques used for recognition and classification of planktonic organisms (segmentation, shape feature extraction, pigment signature determination and neural network grouping) will be described. An introduction and overview of the topic, its current state and indications of future directions the field is expected to take will be provided, organizing the review for both experts and researchers new to the field.

Inelastic hyperspectral Scheimpflug lidar for microalgae classification and quantification

An inelastic hyperspectral Scheimpflug lidar system was developed for microalgae classification and quantification. The correction for the refraction at the air-glass-water interface was established, making our system suitable for aquatic environments. The fluorescence spectrum of microalgae was extracted by principal component analysis, and seven species of microalgae from different phyla have been classified. It was verified that when the cell density of Phaeocystis globosa was in the range of ${{1}}{{{0}}^4}\sim{{1}}{{{0}}^6}\;{\rm{cell}}\;{\rm{m}}{{\rm{L}}^{- 1}}$, the cell density had a linear relationship with the fluorescence intensity. The experimental results show our system can identify and quantify microalgae, with application prospects for microalgae monitoring in the field environment and early warning of red tides or algal blooms.

Identification of spectral signature for in situ real-time monitoring of smoltification

We describe the use of an optical hyperspectral sensing technique to identify the smoltification status of Atlantic salmon (Salmo salar) based on spectral signatures, thus potentially providing smolt producers with an additional tool to verify the osmoregulatory state of salmon. By identifying whether a juvenile salmon is in the biological freshwater stage (parr) or has adapted to the seawater stage (smolt) before transfer to sea, negative welfare impacts and subsequent mortality associated with failed or incorrect identification may be reduced. A hyperspectral imager has been used to collect data in two water flow-through and one recirculating production site in parallel with the standard smoltification evaluations applied at these sites. The results from the latter have been used as baseline for a machine-learning algorithm trained to identify whether a fish was parr or smolt based on its spectral signature. The developed method correctly classified fish in 86% to 100% of the cases for individual sites, and had an overall average classification accuracy of 90%, thus indicating that analysis of spectral signatures may constitute a useful tool for smoltification monitoring.

Classification, identification, and growth stage estimation of microalgae based on transmission hyperspectral microscopic imaging and machine learning

A transmission hyperspectral microscopic imager (THMI) that utilizes machine learning algorithms for hyperspectral detection of microalgae is presented. The THMI system has excellent performance with spatial and spectral resolutions of 4 µm and 3 nm, respectively. We performed hyperspectral imaging (HSI) of three species of microalgae to verify their absorption characteristics. Transmission spectra were analyzed using principal component analysis (PCA) and peak ratio algorithms for dimensionality reduction and feature extraction, and a support vector machine (SVM) model was used for classification. The average accuracy, sensitivity and specificity to distinguish one species from the other two species were found to be 94.4%, 94.4% and 97.2%, respectively. A species identification experiment for a group of mixed microalgae in solution demonstrates the usability of the classification method. Using a random forest (RF) model, the growth stage in a phaeocystis growth cycle cultivated under laboratory conditions was predicted with an accuracy of 98.1%, indicating the feasibility to evaluate the growth state of microalgae through their transmission spectra. Experimental results show that the THMI system has the capability for classification, identification and growth stage estimation of microalgae, with strong potential for in-situ marine environmental monitoring and early warning detection applications.