Assessment of Compressive Raman versus Hyperspectral Raman for Microcalcification Chemical Imaging

Neural Network-Based Filter Design for Compressive Raman Classification of Cells

Cell-based therapies are bound to revolutionize medicine, but significant technical hurdles must be overcome before wider adoption. In particular, nondestructive, label-free methods to characterize cells in real time are needed to optimize the production process and improve quality control. Raman spectroscopy, which provides a fingerprint of a cell's chemical composition, would be an ideal modality but is too slow for high-throughput applications. Compressive Raman techniques, which measure only linear combinations of Raman intensities, can be fast but require careful optimization to deliver high performance. Here, we develop a neural network model to identify optimal parameters for a compressive sensing scheme that reduces measurement time by 2 orders of magnitude. In a data set containing Raman spectra of three different cell types, it achieves up to 90% classification accuracy using only five linear combinations of Raman intensities. Our method thus unlocks the power of Raman spectroscopy for the characterization of cell products.

Fast compressive Raman micro-spectroscopy to image and classify microplastics from natural marine environment

The fast and reliable detection of micron-sized plastic particles from the natural marine environment is an important topic that is mostly addressed using spontaneous Raman spectroscopy. Due to the long (>tens of ms) integration time required to record a viable Raman signal, measurements are limited to a single point per microplastic particle or require very long acquisition times (up to tens of hours). In this work, we develop, validate, and demonstrate a compressive Raman technology using binary spectral filters and single-pixel detection that can image and classify six types of marine microplastic particles over an area of 1 mm2 with a pixel dwell time down to 1.75 ms/pixel and a spatial resolution of 1 µm. This is x10-100 faster than reported in previous studies.

Optimal trade-off filters for compressed Raman classification and spectrum reconstruction

Compressed Raman spectroscopy is a promising technique for fast chemical analysis. In particular, classification between species with known spectra can be performed with measures acquired through a few binary filters. Moreover, it is possible to reconstruct spectra by using enough filters. As classification and reconstruction are competing, designing filters allowing one to perform both tasks is challenging. To tackle this problem, we propose to build optimal trade-off filters, i.e., filters so that there exist no filters achieving better performance in both classification and reconstruction. With this approach, users get an overview of reachable performance and can choose the trade-off most fitting their application.

Optical system design of a DMD-SHS combined modulation interference spectrometer

Digital micromirror device (DMD) and spatial heterodyne spectroscopy (SHS) combined modulation interference spectroscopy (DMD-SHS) introduces a DMD for the secondary modulation of interferometric data to achieve a Hadamard transform. DMD-SHS can improve the performance index of the spectrometer in terms of the SNR, dynamic range, and spectral bandwidth, while retaining the advantages of a conventional SHS. The DMD-SHS optical system is more complex than a traditional SHS, which places more demands on the optical system's spatial layout and the optical components' performance. According to the DMD-SHS modulation mechanism, the functions of the main components were analyzed, and their design requirements were determined. Based on the potassium spectra detection, a DMD-SHS experimental device was designed. The potassium lamp and integrating sphere detection experiments demonstrated the detection capability of the DMD-SHS experimental device with a spectral resolution of 0.0327 nm and a spectral range of 763.66∼771.25n m, which thoroughly verified the feasibility of DMD and SHS combined modulation interference spectroscopy.

Raman microspectroscopy based TNM staging and grading of breast cancer

The tumor-node-metastasis (TNM) system is the most common way that doctors determine the anatomical extent of cancer on the basis of clinical and pathological criteria. In this study, a spectral histopathological study has been carried out to bridge Raman micro spectroscopy with the breast cancer TNM system. A total of seventy breast tissue samples, including healthy tissue, early, middle, and advanced cancer, were investigated to provide detailed insights into compositional and structural variations that accompany breast malignant evolution. After evaluating the main spectral variations in all tissue types, the generalized discriminant analysis (GDA) pathological diagnostic model was established to discriminate the TNM staging and grading information. Moreover, micro-Raman images were reconstructed by K-means clustering analysis (KCA) for visualizing the lobular acinar in healthy tissue and ductal structures in all early, middle and advanced breast cancer tissue groups. While, univariate imaging techniques were adapted to describe the distribution differences of biochemical components such as tryptophan, β-carotene, proteins, and lipids in the scanned regions. The achieved spectral histopathological results not only established a spectra-structure correlations via tissue biochemical profiles but also provided important data and discriminative model references for in vivo Raman-based breast cancer diagnosis.

Unsupervised segmentation of biomedical hyperspectral image data: tackling high dimensionality with convolutional autoencoders

Information about the structure and composition of biopsy specimens can assist in disease monitoring and diagnosis. In principle, this can be acquired from Raman and infrared (IR) hyperspectral images (HSIs) that encode information about how a sample's constituent molecules are arranged in space. Each tissue section/component is defined by a unique combination of spatial and spectral features, but given the high dimensionality of HSI datasets, extracting and utilising them to segment images is non-trivial. Here, we show how networks based on deep convolutional autoencoders (CAEs) can perform this task in an end-to-end fashion by first detecting and compressing relevant features from patches of the HSI into low-dimensional latent vectors, and then performing a clustering step that groups patches containing similar spatio-spectral features together. We showcase the advantages of using this end-to-end spatio-spectral segmentation approach compared to i) the same spatio-spectral technique not trained in an end-to-end manner, and ii) a method that only utilises spectral features (spectral k-means) using simulated HSIs of porcine tissue as test examples. Secondly, we describe the potential advantages/limitations of using three different CAE architectures: a generic 2D CAE, a generic 3D CAE, and a 2D convolutional encoder-decoder architecture inspired by the recently proposed UwU-net that is specialised for extracting features from HSI data. We assess their performance on IR HSIs of real colon samples. We find that all architectures are capable of producing segmentations that show good correspondence with HE stained adjacent tissue slices used as approximate ground truths, indicating the robustness of the CAE-driven spatio-spectral clustering approach for segmenting biomedical HSI data. Additionally, we stress the need for more accurate ground truth information to enable a precise comparison of the advantages offered by each architecture.

Sub-Nanosecond Digital Signal Processing of Photomultiplier Tube Response Enabling Multiphoton Counting in Raman Spectroscopy

This paper introduces a novel approach to achieve multiple photon counting for Raman spectroscopy. The multiphoton counting process is made possible by recording and analyzing the photomultiplier tube response to each pulse of a pulsed laser in a time-resolved Raman spectroscopy system. Conventional Raman spectroscopy typically considers photon arrivals as binary events assessed by a single threshold. Hence, the conventional algorithm ignores the fact that multiple photons could arrive within the same response, sacrificing potential signal gain. In this work, a high-speed data acquisition system and multiple threshold digital signal processing counting algorithm are employed to facilitate multiphoton counting, a data processing approach that differentiates photon arrival events by amplitude and time and contributes to improved Raman detection sensitivity. The multiphoton counting algorithm enables lower concentration detection, greater sensitivity, shortens experiment duration, and improves noise rejection. Results from analyses of aqueous solutions of nitrate, isopropanol, and rhodamine 6G demonstrate the versatility and effectiveness of this algorithm. The algorithm increased system sensitivity by ∼ 2.0-, 2.0-, and 3.1-fold, compared to traditional single-threshold analyses of the same data for tests performed on nitrate, isopropanol, and rhodamine 6G, respectively. Results also demonstrated that the multiphoton counting algorithm increases the upper analysis limit for high Raman-yield compounds, shifting the saturation threshold to a higher concentration in typical concentration versus intensity calibration curves.

Multi-threshold splitting tree algorithm to reduce the number of filters in programmable hyperspectral imaging for fast multi-target classification

Programmable hyperspectral imaging is a promising and efficient technique for fast target classification by coding hyperspectral post-processing algorithms as spectral transmittances, which enables such post-processing to be directly performed by special optical dispersive element during the process of optical imaging. Compared with conventional hyperspectral imaging and post-processing techniques, it shows significant advantages of fast image acquisition, post-processing free, and a much lower load of data transmission and storage. However, when multi-target classification tasks are encountered, the speed would decrease seriously due to the requirement of a large number of filters. In this study, a novel splitting strategy is proposed to reduce the number of filters in programmable hyperspectral imaging for fast multi-target classification while maintaining the classification performance. Numerical simulation experiments were performed on six publicly available hyperspectral data sets. Compared with the conventional splitting strategies, the proposed splitting strategy can reduce the number of filters by 25% to 80% and achieve similar classification performance, which is of great significance to improve the speed of multi-target classification with programmable hyperspectral imaging technique.

Compressed Raman method combining classification and estimation of spectra with optimized binary filters

Compressed Raman methods allow classification between known chemical species with only a few measurements through binary filters. We propose a methodology for binary filter optimization, in which filters are modified at each pixel so that classification can still be achieved pixel by pixel with a few measurements acquired in parallel, while retaining the ability to reconstruct a full spectrum when combining measurements from several pixels. This approach is robust to intensity variations between pixels. It relies on a generalized Bhattacharyya bound and on the Cramér-Rao bound to tailor filters with optimized performance.

Time-correlated single photon Raman spectroscopy at megahertz repetition rates

Significant improvements in time-correlated single photon counting (TCSPC) Raman spectroscopy acquisition times can be achieved through exploitation of megahertz (MHz) laser repetition rates. We have developed a TCSPC Raman spectroscopy system based on a high peak power (>40W) pulsed laser, a high pulse repetition rate (40 MHz), a custom f/1.5 spectrometer, and a 512 spectral channel × 16 time bin single photon avalanche diode line sensor. We report millisecond Raman spectrum acquisition times, a peak Raman count rate of 104 kcps, and a linewidth aggregated count rate of 440 kcps with a diamond sample. This represents a three-order-of-magnitude increase in measured Raman count rate in comparison with a 104 kHz pulsed laser operating at 300 W and a four-order-of-magnitude increase over a 0.1 W pulsed laser operating at 40 MHz. A Raman-to-fluorescence ratio of 4.76 is achieved with a sesame oil sample at a 20 MHz repetition rate. Achieving high count rates and Raman-to-fluorescence ratios unlocks the potential of combined Raman/fluorescence lifetime spectroscopy for imaging and other short acquisition time applications.

Sub-Surface Molecular Analysis and Imaging in Turbid Media Using Time-Gated Raman Spectral Multiplexing

Obtaining molecular information deeper within optically turbid samples is valuable in many applications. However, in many cases this is challenging, in particular when the sample elicits strong laser-induced fluorescence emission. Here, we investigated the use of time-gated and micro-spatially offset Raman spectroscopy (micro-SORS) based on spectral multiplexing detection to obtain sub-surface molecular analysis and imaging for both fluorescing and non-fluorescing samples. The multiplexed spectral detection achieved with a digital micromirror device (DMD) allowed fast acquisition of the time-gated signals to enable three-dimensional Raman mapping (raster scanning in the lateral x,y plane and using time-of-flight calibration for the axial z-direction). Sub-millimeter resolution molecular depth mapping was achieved with dwell times on the order of seconds per pixel. To suppress fluorescence backgrounds and enhance Raman bands, time-gated Raman spectroscopy was combined with micro-SORS to recover Raman signals of red pigments placed behind a layer of optically turbid material. Using a defocusing micro-SORS approach, both fluorescence and Raman signals from the surface layers were further suppressed, which enhanced the Raman signals from the deeper sublayers containing the pigment. These results demonstrate that time-gated Raman spectroscopy based on spectral multiplexed detection, and in combination with micro-SORS, is a powerful technique for sub-surface molecular analysis and imaging, which may find practical applications in medical imaging, cultural heritage, forensics, and industry.

Line-scan compressive Raman imaging with spatiospectral encoding

We report a line-scanning imaging modality of compressive Raman technology with a single-pixel detector. The spatial information along the illumination line is encoded onto one axis of a digital micromirror device, while spectral coding masks are applied along the orthogonal direction. We demonstrate imaging and classification of three different chemical species.

Low resolution Raman: the impact of spectral resolution on limit of detection and imaging speed in hyperspectral imaging

The majority of problems in analytical Raman spectroscopy are mathematically over-determined, where many more spectral variables are measured than analytic outputs (such as chemical concentrations) are calculated. Thus, to improve spectral throughput and simplify system design, some researchers have explored the use of low resolution Raman systems for cell or tissue classification, achieving accuracy independent of spectral resolution. However, the tradeoffs inherent in this approach have not been systematically studied. Here, we theoretically and experimentally explore the relationship between spectral resolution and analytical error. We show that decreased spectral resolution leads to spectral signal-to-noise ratio and therefore more reliable results and lower limits of detection for equivalent integration times in blind unmixing of hyperspectral images. Our theoretical analysis demonstrates that the primary benefit of low resolution Raman spectroscopy is in overcoming detector noise (such as thermal or electronic noise). Therefore, the benefits are most pronounced when utilizing lower-grade, uncooled detectors. Therefore, using a low-cost CMOS camera we experimentally demonstrate the ability of low resolution Raman spectroscopy to achieve substantially improved imaging performance compared to fully-resolved Raman spectral imaging, paving the way for cost-effective, pervasive Raman spectroscopy.

Diagnostic prospects and preclinical development of optical technologies using gold nanostructure contrast agents to boost endogenous tissue contrast

Numerous developments in optical biomedical imaging research utilizing gold nanostructures as contrast agents have advanced beyond basic research towards demonstrating potential as diagnostic tools; some of which are translating into clinical applications. Recent advances in optics, lasers and detection instrumentation along with the extensive, yet developing, knowledge-base in tailoring the optical properties of gold nanostructures has significantly improved the prospect of near-infrared (NIR) optical detection technologies. Of particular interest are optical coherence tomography (OCT), photoacoustic imaging (PAI), multispectral optoacoustic tomography (MSOT), Raman spectroscopy (RS) and surface enhanced spatially offset Raman spectroscopy (SESORS), due to their respective advancements. Here we discuss recent technological developments, as well as provide a prediction of their potential to impact on clinical diagnostics. A brief summary of each techniques' capability to distinguish abnormal (disease sites) from normal tissues, using endogenous signals alone is presented. We then elaborate on the use of exogenous gold nanostructures as contrast agents providing enhanced performance in the above-mentioned techniques. Finally, we consider the potential of these approaches to further catalyse advances in pre-clinical and clinical optical diagnostic technologies.

Optical biomedical imaging research utilising gold nanostructures as contrast agents has advanced beyond basic science, demonstrating potential in various optical diagnostic tools; some of which are currently translating into clinical applications.

Imaging and quantifying drug delivery in skin - Part 2: Fluorescence andvibrational spectroscopic imaging methods

Understanding the delivery and diffusion of topically-applied drugs on human skin is of paramount importance in both pharmaceutical and cosmetics research. This information is critical in early stages of drug development and allows the identification of the most promising ingredients delivered at optimal concentrations to their target skin compartments. Different skin imaging methods, invasive and non-invasive, are available to characterize and quantify the spatiotemporal distribution of a drug within ex vivo and in vivo human skin. The first part of this review detailed invasive imaging methods (autoradiography, MALDI and SIMS). This second part reviews non-invasive imaging methods that can be applied in vivo: i) fluorescence (conventional, confocal, and multiphoton) and second harmonic generation microscopies and ii) vibrational spectroscopic imaging methods (infrared, confocal Raman, and coherent Raman scattering microscopies). Finally, a flow chart for the selection of imaging methods is presented to guide human skin ex vivo and in vivo drug delivery studies.

Spatial light-modulated stimulated Raman scattering (SLM-SRS) microscopy for rapid multiplexed vibrational imaging

High speed imaging is pre-requisite for monitoring of dynamic processes in biological events. Here we report the development of a unique spatial light-modulated stimulated Raman scattering (SLM-SRS) microscopy that tailors the broadband excitation beam with sparse-sampling masks designed for rapid multiplexed vibrational imaging to monitor real-time cancer treatment effects and in vivo transport of drug solvent.

Methods: We design an optimal mask pattern that enables selection of predominant windows in SRS spectrum for collective excitation at the highest possible peak power, thus providing an improved signal-to-noise ratio (SNR) without compromise of chemical specificity. The mask pattern generated is applied to the broad excitation beam using a flexible spatial light modulator. The SLM module further offers complementary function whereby rapid scanning of SRS spectrum can be facilitated prior to the mask generation, thereby making the SLM-SRS system a stand-alone imaging platform.

Results: We demonstrate that SLM-SRS microscopy permits rapid multiplexed SRS imaging of polystyrene and polymethyl methacrylate beads in Brownian motion in dimethyl sulfoxide (DMSO) at 70 ms intervals without motion artiacts. We further apply SLM-SRS to monitor the therapeautic effect of mild alkaline solution on cancer cells, which shows immediate apoptotic response. Finally, we visualize in vivo penetration of DMSO into the plant tissue and evaluate acute toxicity of DMSO on cellulose and proteins within the tissue.

Conclusion: We develop novel SLM-SRS microscopy and affirm its broad applicability for rapid monitoring of dynamic biological processes at the subcellular and molecular level.

Compressed Raman classification method with upper-bounded error probability

Classification of different species with Raman measurements is analyzed when a total of exactly $ N $N photons are detected with binary filtered Raman spectra instead of fixing the measuring time. The optimal classification method for this problem leads to classification error probabilities upper-bounded by the Bhattacharyya bound and that are invariant to the multiplication of the spectrum intensities by an unknown factor. Furthermore, it is shown that this approach can be implemented with a number of binary filters smaller than the number of species to discriminate.

Applications of Spatial Light Modulators in Raman Spectroscopy

Advances in consumer display screen technologies have historically been adapted by researchers across the fields of optics as they can be used as electronically controlled spatial light modulators (SLMs) for a variety of uses. The performance characteristics of such SLM devices based on liquid crystal (LC) and digital micromirror device (DMD) technologies, in particular, has developed to the point where they are compatible with increasingly sensitive instrumental applications, for example, Raman spectroscopy. Spatial light modulators provide additional flexibility, from modulation of the laser excitation (including multiple laser foci patterns), manipulation of microscopic samples (optical trapping), or selection of sampling volume (adaptive optics or spatially offset Raman spectroscopy), to modulation in the spectral domain for high-resolution spectral filtering or multiplexed/compressive fast detection. Here, we introduce the benefits of different SLM devices as a part of Raman instrumentation and provide a variety of recent example applications which have benefited from their incorporation into a Raman system.

Bhattacharyya bound for Raman spectrum classification with a couple of binary filters

Bhattacharyya bounds of classification error probability between two species with Raman and binary compressed Raman measurements limited by Poisson photon noise are analyzed. They exhibit the relevant physical parameters and lead to a simple expression of a minimal number of photons necessary to upper bound the optimal classification error probability.

Compressive Raman imaging with spatial frequency modulated illumination

We report a line scanning imaging modality of compressive Raman technology with spatial frequency modulated illumination using a single pixel detector. We demonstrate the imaging and classification of three different chemical species at line scan rates of 40 Hz.

Recent Trends in Compressive Raman Spectroscopy Using DMD-Based Binary Detection

The collection of high-dimensional hyperspectral data is often the slowest step in the process of hyperspectral Raman imaging. With the conventional array-based Raman spectroscopy acquiring of chemical images could take hours to even days. To increase the Raman collection speeds, a number of compressive detection (CD) strategies, which simultaneously sense and compress the spectral signal, have recently been demonstrated. As opposed to conventional hyperspectral imaging, where full spectra are measured prior to post-processing and imaging CD increases the speed of data collection by making measurements in a low-dimensional space containing only the information of interest, thus enabling real-time imaging. The use of single channel detectors gives the key advantage to CD strategy using optical filter functions to obtain component intensities. In other words, the filter functions are simply the optimized patterns of wavelength combinations characteristic of component in the sample, and the intensity transmitted through each filter represents a direct measure of the associated score values. Essentially, compressive hyperspectral images consist of 'score' pixels (instead of 'spectral' pixels). This paper presents an overview of recent advances in compressive Raman detection designs and performance validations using a DMD based binary detection strategy.

Fast Raman spectral mapping of highly fluorescing samples by time-gated spectral multiplexed detection

We present a time-gated Raman micro-spectroscopy technique suitable for fast Raman mapping of samples eliciting large laser-induced fluorescence backgrounds. To achieve the required time resolution for effective fluorescence rejection, a picosecond pulsed laser and a single-photon avalanche diode were used. A module consisting of a spectrometer, digital micromirror device, and two prisms was used for high-resolution spectral filtering and multiplexing, which is required for a high chemical specificity and short integration times. With this instrument, we demonstrated time-gated Raman imaging of highly fluorescent samples, achieving acquisition times as short as 3 min for 40×40 pixel resolution images.