Optical Scatter Imaging with a digital micromirror device

Engineering pupil function for optical adversarial attacks

Adversarial attacks inject imperceptible noise to images to deteriorate the performance of deep image classification models. However, most of the existing studies consider attacks in the digital (pixel) domain where an image acquired by an image sensor with sampling and quantization is recorded. This paper, for the first time, introduces a scheme for optical adversarial attack, which physically alters the light field information arriving at the image sensor so that the classification model yields misclassification. We modulate the phase of the light in the Fourier domain using a spatial light modulator placed in the photographic system. The operative parameters of the modulator for adversarial attack are obtained by gradient-based optimization to maximize cross-entropy and minimize distortion. Experiments based on both simulation and a real optical system demonstrate the feasibility of the proposed optical attack. We show that our attack can conceal perturbations in the image more effectively than the existing pixel-domain attack. It is also verified that the proposed attack is completely different from common optical aberrations such as spherical aberration, defocus, and astigmatism in terms of both perturbation patterns and classification results.

Rapid detection and identification of bacteria directly from whole blood with light scattering spectroscopy based biosensor

Bacterial infections are one of the major causes of death worldwide. The identification of a bacterial species that is the source of an infection generally takes a long time, and often exceeds the treatment window for seriously ill patients. Many of these deaths are preventable if the bacterial species can be identified quickly. Here we present an optical spectroscopic method for rapid detection and identification of bacteria directly from whole blood using a light scattering spectroscopy technique. This technique was originally developed to detect pre-cancerous changes in epithelial tissues, characterize changes in tissue on the cellular scale, and characterize biological structures comparable to or smaller than a single wavelength. We demonstrate here that not only can an inexpensive light scattering spectroscopy-based biosensor rapidly detect and identify four bacteria species in the blood, responsible for the majority of death causing infections, but that species-level identification can potentially be made based on approximately one thousand bacterial cells per milliliter of blood. Observing entire colonies or performing susceptibility testing is therefore not required.

Dark Field and Coherent Anti-Stokes Raman (DF-CARS) Imaging of Cell Uptake of Core-Shell, Magnetic-Plasmonic Nanoparticles

Magnetic-plasmonic, Fe₃O₄-Au, core-shell nanoparticles are popular in many applications, most notably in therapeutics and diagnostics, and thus, the imaging of these nanostructures in biological samples is of high importance. These nanostructures are typically imaged in biological material by dark field scatter imaging, which requires an even distribution of nanostructures in the sample and, therefore, high nanoparticle doses, potentially leading to toxicology issues. Herein, we explore the nonlinear optical properties of magnetic nanoparticles coated with various thicknesses of gold using the open aperture z-scan technique to determine the nonlinear optical properties and moreover, predict the efficacy of the nanostructures in nonlinear imaging. We find that the magnetic nanoparticles coated with gold nanoseeds and thinner gold shells (ca. 4 nm) show the largest nonlinear absorption coefficient β and imaginary part of the third-order susceptibility Im χ⁽³⁾, suggesting that these nanostructures would be suitable contrast agents. Next, we combine laser dark field microscopy and epi-detected coherent anti-Stokes Raman (CARS) microscopy to image the uptake of magnetic-plasmonic nanoparticles in human pancreatic cancer cells. We show the epi-detected CARS technique is suitable for imaging of the magnetic-plasmonic nanoparticles without requiring a dense distribution of nanoparticles. This technique achieves superior nanoparticle contrasting over both epi-detected backscatter imaging and transmission dark field imaging, while also attaining label-free chemical contrasting of the cell. Lastly, we show the high biocompatibility of the Fe₃O₄ nanoparticles with ca. 4-nm thick Au shell at concentrations of 10-100 µg/mL.

Nanometer precise red blood cell sizing using a cost-effective quantitative dark field imaging system

Because of the bulk, complexity, calibration requirements, and need for operator training, most current flow-based blood counting devices are not appropriate for field use. Standard imaging methods could be much more compact, inexpensive, and with minimal calibration requirements. However, due to the diffraction limit, imaging lacks the nanometer precision required to measure red blood cell volumes. To address this challenge, we utilize Mie scattering, which can measure nanometer-scale morphological information from cells, in a dark-field imaging geometry. The approach consists of a custom-built dark-field scattering microscope with symmetrically oblique illumination at a precisely defined angle to record wide-field images of diluted and sphered blood samples. Scattering intensities of each cell under three wavelengths are obtained by segmenting images via digital image processing. These scattering intensities are then used to determine size and hemoglobin information via Mie theory and machine learning. Validation on 90 clinical blood samples confirmed the ability to obtain mean corpuscular volume (MCV), mean corpuscular hemoglobin concentration (MCHC), and red cell distribution width (RDW) with high accuracy. Simulations based on historical data suggest that an instrument with the accuracy achieved in this study could be used for widespread anemia screening.

Light scattering methods for tissue diagnosis

Light scattering has become a common biomedical research tool, enabling diagnostic sensitivity to myriad tissue alterations associated with disease. Light-tissue interactions are particularly attractive for diagnostics due to the variety of contrast mechanisms that can be used, including spectral, angle-resolved, and Fourier-domain detection. Photonic diagnostic tools offer further benefit in that they are non-ionizing, non-invasive, and give real-time feedback. In this review, we summarize recent innovations in light scattering technologies, with a focus on clinical achievements over the previous ten years.

High-throughput label-free flow cytometry based on matched-filter compressive imaging

We present a fast label-free computational flow cytometer based on a strategy of compressive imaging. Scattered light from flowing objects is sub-divided into user-defined basis patterns by a deformable mirror and routed to different detectors associated with each pattern. The patterns can be optimized to be matched to the object features of interest, thus facilitating object identification and separation. Compared to conventional scanning flow cytometers, our technique provides increased information capacity without sacrificing flow velocity. Unique features of our matched-filter strategy are that it can simultaneously probe multiple objects throughout large fields of view with long depths of field. In our proof-of-concept demonstrations, we achieve throughputs of over 10,000 particles/s, working at flow velocities of over 1m/s.

Digital micro-mirror device-based detector for particle-sizing instruments via Fraunhofer diffraction

In this paper, a digital micro-mirror device (DMD)-based detector is proposed for the detection of light intensity in particle-sizing instruments using Fraunhofer diffraction. The detector consists of only one photodiode, which eliminates the distortions caused by the nonuniformity of the detector arrays used in traditional instruments. The center of the diffraction pattern was accurately located to distribute the optimized arc-shaped mirror arrays for the intensity detection. Both simulated and experimental results showed that the proposed detector was superior to the classical one as it was less sensitive to noise than the detector arrays used in traditional systems.

DMD-based LED-illumination super-resolution and optical sectioning microscopy

Super-resolution three-dimensional (3D) optical microscopy has incomparable advantages over other high-resolution microscopic technologies, such as electron microscopy and atomic force microscopy, in the study of biological molecules, pathways and events in live cells and tissues. We present a novel approach of structured illumination microscopy (SIM) by using a digital micromirror device (DMD) for fringe projection and a low-coherence LED light for illumination. The lateral resolution of 90 nm and the optical sectioning depth of 120 μm were achieved. The maximum acquisition speed for 3D imaging in the optical sectioning mode was 1.6×10⁷ pixels/second, which was mainly limited by the sensitivity and speed of the CCD camera. In contrast to other SIM techniques, the DMD-based LED-illumination SIM is cost-effective, ease of multi-wavelength switchable and speckle-noise-free. The 2D super-resolution and 3D optical sectioning modalities can be easily switched and applied to either fluorescent or non-fluorescent specimens.

Measurement of object structure from size-encoded images generated by optically-implemented Gabor filters

We use optical Fourier processing based on two dimensional (2D) Gabor filters to obtain size-encoded images which depict with 20nm sensitivity to size while preserving a 0.36μm spatial resolution, the spatial distribution of structural features within transparent objects. The size of the object feature measured at each pixel in the encoded image is determined by the optimal Gabor filter period, S(max), that maximizes the scattering signal from that location in the object. We show that S(max) (in μm) depends linearly on feature size (also in μm) over a size range from 0.11μm to 2μm. This linear response remains largely unchanged when the refractive index ratio is varied and can be predicted from numerical simulations of Gabor-filtered light scattering. Pixel histograms of the size-encoded images of isolated spheres and diatoms were used to generate highly resolved size distributions ("size spectra") exhibiting sharp peaks characterizing the known major structural features within the studied objects. Dynamic signal associated with changes in selected feature sizes within living cells is also demonstrated. Taken together, our data suggest that a label-free, direct and objective measurement of sample structure is enabled by the size-encoded images and associated pixel histograms generated from a calibrated optical processing microscope based on Gabor filtering.

Quantitative Carré differential interference contrast microscopy to assess phase and amplitude

We present a method of using an unmodified differential interference contrast microscope to acquire quantitative information on scatter and absorption of thin tissue samples. A simple calibration process is discussed that uses a standard optical wedge. Subsequently, we present a phase-stepping procedure for acquiring phase gradient information exclusive of absorption effects. The procedure results in two-dimensional maps of the local angular (polar and azimuthal) ray deviation. We demonstrate the calibration process, discuss details of the phase-stepping algorithm, and present representative results for a porcine skin sample.

Quantifying subcellular dynamics in apoptotic cells with two-dimensional Gabor filters

We demonstrate an optical Fourier filtering method which can be used to characterize subcellular morphology during dynamic cellular function. In this paper, our Fourier filters were based on two-dimensional Gabor elementary functions, which can be tuned to sense directly object size and orientation. We utilize this method to quantify changes in mitochondrial and nuclear structure during the first three hours of apoptosis. We find that the technique is sensitive to a decrease in particle orientation consistent with apoptosis-induced mitochondrial fragmentation. The scattering signal changes were less pronounced in the nucleus and the remainder of the cytoplasm. Particles in these regions were less oriented than mitochondria and did not change orientation significantly.

Microscopic imaging and spectroscopy with scattered light

Optical contrast based on elastic scattering interactions between light and matter can be used to probe cellular structure, cellular dynamics, and image tissue architecture. The quantitative nature and high sensitivity of light scattering signals to subtle alterations in tissue morphology, as well as the ability to visualize unstained tissue in vivo, has recently generated significant interest in optical-scatter-based biosensing and imaging. Here we review the fundamental methodologies used to acquire and interpret optical scatter data. We report on recent findings in this field and present current advances in optical scatter techniques and computational methods. Cellular and tissue data enabled by current advances in optical scatter spectroscopy and imaging stand to impact a variety of biomedical applications including clinical tissue diagnosis, in vivo imaging, drug discovery, and basic cell biology.

Optical scatter changes at the onset of apoptosis are spatially associated with mitochondria

We combine optical scatter imaging (OSI) with fluorescence imaging of mitochondria to investigate the spatial relationship between the optical scatter signal and the location and structure of mitochondria within endothelial cells undergoing apoptosis. The OSI data corroborate our previous results showing a decrease in the intensity ratio of wide-to-narrow angle scatter [optical scatter image ratio (OSIR)] during the first 60 min of apoptosis. In addition, we find here that this is followed by an increase in OSIR concurrent with mitochondrial fragmentation. We demonstrate that the dynamic change in light scattering is spatially associated with subcellular regions containing fluorescently labeled mitochondria, and remains absent from adjacent nonfluorescent regions dominated by other organelles. These results lend strong support to the hypothesis that mitochondria act as the source of the optical scatter changes measured at the onset of apoptosis.

Alterations in the characteristic size distributions of subcellular scatterers at the onset of apoptosis: effect of Bcl-xL and Bax/Bak

Optical scatter imaging is used to estimate organelle size distributions in immortalized baby mouse kidney cells treated with 0.4 microM staurosporine to induce apoptosis. The study comprises apoptosis competent iBMK cells (W2) expressing the proapoptotic proteins Bax/Bak, apoptosis resistant Bax/Bak null cells (D3), and W2 and D3 cells expressing yellow fluorescent protein (YFP) or YFP fused to the antiapoptotic protein Bcl-x(L) (YFP-Bcl-x(L)). YFP expression is diffuse within the transfected cells, while YFP-Bcl-x(L) is localized to the mitochondria. Our results show a significant increase in the mean subcellular particle size from approximately 1.1 to 1.4 microm in both Bax/Bak expressing and Bax/Bak null cells after 60 min of STS treatment compared to DMSO-treated control cells. This dynamic is blocked by overexpression of YFP-Bcl-x(L) in Bax/Bak expressing cells, but is less significantly inhibited by YFP-Bcl-x(L) in Bax/Bak null cells. Our data suggest that the increase in subcellular particle size at the onset of apoptosis is modulated by Bcl-x(L) in the presence of Bax/Bak, but it occurs upstream of the final commitment to programmed cell death. Mitochondrial localization of YFP-Bcl-x(L) and the finding that micron-sized particles give rise to the scattering signal further suggest that alterations in mitochondrial morphology may underlie the observed changes in light scattering.

Optical scatter microscopy based on two-dimensional Gabor filters

We demonstrate a microscopic instrument that can measure subcellular texture arising from organelle morphology and organization within unstained living cells. The proposed instrument extends the sensitivity of label-free optical microscopy to nanoscale changes in organelle size and shape and can be used to accelerate the study of the structure-function relationship pertaining to organelle dynamics underlying fundamental biological processes, such as programmed cell death or cellular differentiation. The microscope can be easily implemented on existing microscopy platforms, and can therefore be disseminated to individual laboratories, where scientists can implement and use the proposed methods with unrestricted access. The proposed technique is able to characterize subcellular structure by observing the cell through two-dimensional optical Gabor filters. These filters can be tuned to sense with nanoscale (10's of nm) sensitivity, specific morphological attributes pertaining to the size and orientation of non-spherical subcellular organelles. While based on contrast generated by elastic scattering, the technique does not rely on a detailed inverse scattering model or on Mie theory to extract morphometric measurements. This technique is therefore applicable to non-spherical organelles for which a precise theoretical scatter description is not easily given, and provides distinctive morphometric parameters that can be obtained within unstained living cells to assess their function. The technique is advantageous compared with digital image processing in that it operates directly on the object's field transform rather than the discretized object's intensity. It does not rely on high image sampling rates and can therefore be used to rapidly screen morphological activity within hundreds of cells at a time, thus greatly facilitating the study of organelle structure beyond individual organelle segmentation and reconstruction by fluorescence confocal microscopy of highly magnified digital images of limited fields of view. In this demonstration we show data from a marine diatom to illustrate the methodology. We also show preliminary data collected from living cells to give an idea of how the method may be applied in a relevant biological context.