Fast phase processing in off-axis holography by CUDA including parallel phase unwrapping

Dynamic tracking of onion-like carbon nanoparticles in cancer cells using limited-angle holographic tomography with self-supervised learning

This research presents a novel approach for the dynamic monitoring of onion-like carbon nanoparticles inside colorectal cancer cells. Onion-like carbon nanoparticles are widely used in photothermal cancer therapy, and precise 3D tracking of their distribution is crucial. We proposed a limited-angle digital holographic tomography technique with unsupervised learning to achieve rapid and accurate monitoring. A key innovation is our internal learning neural network. This network addresses the information limitations of limited-angle measurements by directly mapping coordinates to measured data and reconstructing phase information at unmeasured angles without external training data. We validated the network using standard SiO2 microspheres. Subsequently, we reconstructed the 3D refractive index of onion-like carbon nanoparticles within cancer cells at various time points. Morphological parameters of the nanoparticles were quantitatively analyzed to understand their temporal evolution, offering initial insights into the underlying mechanisms. This methodology provides a new perspective for efficiently tracking nanoparticles within cancer cells.

Recent Advances and Current Trends in Transmission Tomographic Diffraction Microscopy

Optical microscopy techniques are among the most used methods in biomedical sample characterization. In their more advanced realization, optical microscopes demonstrate resolution down to the nanometric scale. These methods rely on the use of fluorescent sample labeling in order to break the diffraction limit. However, fluorescent molecules' phototoxicity or photobleaching is not always compatible with the investigated samples. To overcome this limitation, quantitative phase imaging techniques have been proposed. Among these, holographic imaging has demonstrated its ability to image living microscopic samples without staining. However, for a 3D assessment of samples, tomographic acquisitions are needed. Tomographic Diffraction Microscopy (TDM) combines holographic acquisitions with tomographic reconstructions. Relying on a 3D synthetic aperture process, TDM allows for 3D quantitative measurements of the complex refractive index of the investigated sample. Since its initial proposition by Emil Wolf in 1969, the concept of TDM has found a lot of applications and has become one of the hot topics in biomedical imaging. This review focuses on recent achievements in TDM development. Current trends and perspectives of the technique are also discussed.

Multi-derivative method for phase extraction without knowing carrier frequencies in off-axis quantitative phase imaging

We propose a multi-derivative method to reconstruct the phase of transparent objects in off-axis quantitative phase imaging (QPI). By numerically computing first-, second-, and third-order derivatives of the interferogram, we demonstrate that one can extract the quantitative phase information in a straightforward way, without prior knowledge of the carrier frequencies or Fourier transform. In contrast to existing advanced derivative methods, our approach markedly streamlines the alignment and retrieval processes, all without requiring any special prerequisites. This enhancement seamlessly translates into improved reconstruction quality. Furthermore, when compared to cutting-edge Fourier-division-based methods, our technique distinctly accelerates the phase retrieval speed. We verified our method using white-light diffraction phase microscopy and laser off-axis QPI, and the results indicate that our method can allow a fast, high-quality retrieval with frame rates up to 41.6 fps for one- megapixel interferograms on a regular computer.

Dual-wavelength off-axis digital holography in ImageJ: toward real-time phase retrieval using CUDA streams

An ImageJ plug-in is developed to realize automatic real-time phase reconstruction for dual-wavelength digital holography (DH). This plug-in assembles the algorithms, including automatic phase reconstruction based on the division algorithm and post-processing. These algorithms are implemented and analyzed using a CPU and GPU, respectively. To hide the CPU-to-GPU data transfer latency, an optimization scheme using Compute Unified Device Architecture (CUDA) streams is proposed in ImageJ. Experimental results show that the proposed plug-in can perform faster reconstruction for dual-wavelength DH, resulting in frame rates up to 48 fps even for one-megapixel digital holograms on a normal PC. In other words, the proposed plug-in can realize real-time phase reconstruction for dual-wavelength digital holographic videos.

Digital Holographic Microscopy for Label-Free Detection of Leukocyte Alternations Associated with Perioperative Inflammation after Cardiac Surgery

In a prospective observational pilot study on patients undergoing elective cardiac surgery with cardiopulmonary bypass, we evaluated label-free quantitative phase imaging (QPI) with digital holographic microscopy (DHM) to describe perioperative inflammation by changes in biophysical cell properties of lymphocytes and monocytes. Blood samples from 25 patients were investigated prior to cardiac surgery and postoperatively at day 1, 3 and 6. Biophysical and morphological cell parameters accessible with DHM, such as cell volume, refractive index, dry mass, and cell shape related form factor, were acquired and compared to common flow cytometric blood cell markers of inflammation and selected routine laboratory parameters. In all examined patients, cardiac surgery induced an acute inflammatory response as indicated by changes in routine laboratory parameters and flow cytometric cell markers. DHM results were associated with routine laboratory and flow cytometric data and correlated with complications in the postoperative course. In a subgroup analysis, patients were classified according to the inflammation related C-reactive protein (CRP) level, treatment with epinephrine and the occurrence of postoperative complications. Patients with regular courses, without epinephrine treatment and with low CRP values showed a postoperative lymphocyte volume increase. In contrast, the group of patients with increased CRP levels indicated an even further enlarged lymphocyte volume, while for the groups of epinephrine treated patients and patients with complicative courses, no postoperative lymphocyte volume changes were detected. In summary, the study demonstrates the capability of DHM to describe biophysical cell parameters of perioperative lymphocytes and monocytes changes in cardiac surgery patients. The pattern of correlations between biophysical DHM data and laboratory parameters, flow cytometric cell markers, and the postoperative course exemplify DHM as a promising diagnostic tool for a characterization of inflammatory processes and course of disease.

High-speed measurement of mechanical micro-deformations with an extended phase range using dual-wavelength digital holographic interferometry

The implementation of a digital holographic interferometry setup for high-speed micro-deformation measurement is presented. This proposal uses a dual-wavelength recording strategy to reconstruct micro-deformations up to 4.85 µm with no phase wrapping. The numerical processing required to recover the phase maps containing the information of micro-deformations is carried out in a general-purpose computing on graphics processing unit environment to boost its performance. The method completely processes recorded holograms of 1024×1024pixels in 48 ms, i.e., 21 frames per second (FPS) for a single-wavelength acquisition and 96 ms or 11 FPS for dual-wavelength recordings. The method is experimentally evaluated measuring deformations ranging from 0.033 µm to 4.85 µm with no need for phase unwrapping algorithms for an 8 cm diameter aluminum plate in a 110cm² field of view.

Unsupervised feature learning and clustering of particles imaged in raw holograms using an autoencoder

Digital holography is a useful tool to image microscopic particles. Reconstructed holograms give high-resolution shape information that can be used to identify the types of particles. However, the process of reconstructing holograms is computationally intensive and cannot easily keep up with the rate of data acquisition on low-power sensor platforms. In this work, we explore the possibility of performing object clustering on holograms that have not been reconstructed, i.e., images of raw interference patterns, using the latent representations of a deep-learning autoencoder and a self-organizing mapping network in a fully unsupervised manner. We demonstrate this concept on synthetically generated holograms of different shapes, where clustering of raw holograms achieves an accuracy of 94.4%. This is comparable to the 97.4% accuracy achieved using the reconstructed holograms of the same targets. Directly clustering raw holograms takes less than 0.1 s per image using a low-power CPU board. This represents a three-order of magnitude reduction in processing time compared to clustering of reconstructed holograms and makes it possible to interpret targets in real time on low-power sensor platforms. Experiments on real holograms demonstrate significant gains in clustering accuracy through the use of synthetic holograms to train models. Clustering accuracy increased from 47.1% when the models were trained only on the real raw holograms, to 64.1% when the models were entirely trained on the synthetic raw holograms, and further increased to 75.9% when models were trained on the both synthetic and real datasets using transfer learning. These results are broadly comparable to those achieved when reconstructed holograms are used, where the highest accuracy of 70% achieved when clustering raw holograms outperforms the highest accuracy achieved when clustering reconstructed holograms by a significant margin for our datasets.

Real-Time Stain-Free Classification of Cancer Cells and Blood Cells Using Interferometric Phase Microscopy and Machine Learning

We present a method for real-time visualization and automatic processing for detection and classification of untreated cancer cells in blood during stain-free imaging flow cytometry using digital holographic microscopy and machine learning in throughput of 15 cells per second. As a preliminary model for circulating tumor cells in the blood, following an initial label-free rapid enrichment stage based on the cell size, we applied our holographic imaging approach, providing the quantitative optical thickness profiles of the cells during flow. We automatically classified primary and metastatic colon cancer cells, where the two types of cancer cells were isolated from the same individual, as well as four types of blood cells. We used low-coherence off-axis interferometric phase microscopy and a microfluidic channel to image cells during flow quantitatively. The acquired images were processed and classified based on their morphology and quantitative phase features during the cell flow. We achieved high accuracy of 92.56% for distinguishing between the cells, enabling further automatic enrichment and cancer-cell grading from blood. © 2020 International Society for Advancement of Cytometry.

Head-Mounted Display-Based Microscopic Imaging System with Customizable Field Size and Viewpoint

In recent years, the use of microinjections has increased in life science and biotechnology fields; specific examples include artificial insemination and gene manipulation. Microinjections are mainly performed based on visual information; thus, the operator needs high-level skill because of the narrowness of the visual field. Additionally, microinjections are performed as the operator views a microscopic image on a display; the position of the display requires the operator to maintain an awkward posture throughout the procedure. In this study, we developed a microscopic image display apparatus for microinjections based on a view-expansive microscope. The prototype of the view-expansive microscope has problems related to the variations in brightness and focal blur that accompany changes in the optical path length and amount of reflected light. Therefore, we propose the use of a variable-focus device to expand the visual field and thus circumvent the above-mentioned problems. We evaluated the observable area of the system using this variable-focus device. We confirmed that the observable area is 261.4 and 13.9 times larger than that of a normal microscope and conventional view-expansive microscopic system, respectively. Finally, observations of mouse embryos were carried out by using the developed system. We confirmed that the microscopic images can be displayed on a head-mounted display in real time with the desired point and field sizes.

Quantitative phase imaging of cells in a flow cytometry arrangement utilizing Michelson interferometer-based off-axis digital holographic microscopy

We combined Michelson-interferometer-based off-axis digital holographic microscopy (DHM) with a common flow cytometry (FCM) arrangement. Utilizing object recognition procedures and holographic autofocusing during the numerical reconstruction of the acquired off-axis holograms, sharply focused quantitative phase images of suspended cells in flow were retrieved without labeling, from which biophysical cellular features of distinct cells, such as cell radius, refractive index and dry mass, can be subsequently retrieved in an automated manner. The performance of the proposed concept was first characterized by investigations on microspheres that were utilized as test standards. Then, we analyzed two types of pancreatic tumor cells with different morphology to further verify the applicability of the proposed method for quantitative live cell imaging. The retrieved biophysical datasets from cells in flow are found in good agreement with results from comparative investigations with previously developed DHM methods under static conditions, which demonstrates the effectiveness and reliability of our approach. Our results contribute to the establishment of DHM in imaging FCM and prospect to broaden the application spectrum of FCM by providing complementary quantitative imaging as well as additional biophysical cell parameters which are not accessible in current high-throughput FCM measurements.

Fast quality-guided phase unwrapping algorithm through a pruning strategy: applications in dynamic interferometry

The quality-guided phase unwrapping algorithm is one of the most employed spatial algorithms due to its computational efficiency and robustness. It uses a quality map to guide the unwrapping process such that pixels are processed according to their quality values from highest to lowest. Several improvements have been proposed during the last few years with the purpose of using it in time-demanding applications. However, many of the proposals depend on the distribution of the values on the given quality map. In this paper, a novel pruning strategy based on a red-black tree data structure is proposed, whose complexity time is independent of the distribution of the given quality map. We take advantage of the partial ordering of the branches in a red-black tree together with a pruning strategy to speed up the unwrapping process. Experimental results, using real and simulated data, show that the complexity time of our proposal improves the existing quality-guide-based algorithms. Also, a series of interferometric patterns of a time-varying phase distribution experiment have been processed showing that our proposal can be used for real-time applications. The source code of the implemented algorithms is publicly available.

Compact interferometric module for full-field interferometric phase microscopy with low spatial coherence illumination

We propose a compact and external off-axis interferometric module that can achieve interference with low spatial coherence illumination over the entire field of view. The interferometer is easy to align and stable and can be connected to the output of an existing microscope illuminated with a low spatial coherence light source, thus allowing quantitative phase imaging with a low degree of spatial noise. We demonstrate the imaging and the interference properties of the proposed interferometric module and use it for quantitative phase imaging of reflective samples.

Quantitative phase imaging for cell culture quality control

The potential of quantitative phase imaging (QPI) with digital holographic microscopy (DHM) for quantification of cell culture quality was explored. Label-free QPI of detached single cells in suspension was performed by Michelson interferometer-based self-interference DHM. Two pancreatic tumor cell lines were chosen as cellular model and analyzed for refractive index, volume, and dry mass under varying culture conditions. Firstly, adequate cell numbers for reliable statistics were identified. Then, to characterize the performance and reproducibility of the method, we compared results from independently repeated measurements and quantified the cellular response to osmolality changes of the cell culture medium. Finally, it was demonstrated that the evaluation of QPI images allows the extraction of absolute cell parameters which are related to cell layer confluence states. In summary, the results show that QPI enables label-free imaging cytometry, which provides novel complementary integral biophysical data sets for sophisticated quantification of cell culture quality with minimized sample preparation. © 2017 International Society for Advancement of Cytometry.

Quadriwave lateral shearing interferometric microscopy with wideband sensitivity enhancement for quantitative phase imaging in real time

Real-time quantitative phase imaging has tremendous potential in investigating live biological specimens in vitro. Here we report on a wideband sensitivity-enhanced interferometric microscopy for quantitative phase imaging in real time by employing two quadriwave lateral shearing interferometers based on randomly encoded hybrid gratings with different lateral shears. Theoretical framework to analyze the measurement sensitivity is firstly proposed, from which the optimal lateral shear pair for sensitivity enhancement is also derived. To accelerate the phase retrieval algorithm for real-time visualization, we develop a fully vectorized path-independent differential leveling phase unwrapping algorithm ready for parallel computing, and the framerate for retrieving the phase from each pair of two 4 mega pixel interferograms is able to reach 47.85 frames per second. Experiment results demonstrate that the wideband sensitivity-enhanced interferometric microscopy is capable of eliminating all the periodical error caused by spectral leaking problem and reducing the temporal standard deviation to the half level compared with phase directly retrieved by the interferogram. Due to its high adaptability, the wideband sensitivity-enhanced interferometric microscopy is promising in retrofitting existing microscopes to quantitative phase microscopes with high measurement precision and real-time visualization.

Holography with a neutron interferometer

We use a Mach-Zehnder interferometer to perform neutron holography of a spiral phase plate. The object beam passes through a spiral phase plate, acquiring the phase twist characteristic of orbital angular momentum states. The reference beam passes through a fused silica prism, acquiring a linear phase gradient. The resulting hologram is a fork dislocation image, which could be used to reconstruct neutron beams with various orbital angular momenta. This work paves the way for novel applications of neutron holography, diffraction and imaging.

Efficient multiscale phase unwrapping methodology with modulo wavelet transform

Many robust phase unwrapping algorithms are computationally very time-consuming, making them impractical for handling large datasets or real-time applications. In this paper, we propose a generic framework using a novel wavelet transform that can be combined with many types of phase unwrapping algorithms. By inserting reversible modulo operators in the wavelet transform, the number of coefficients that need to be unwrapped is significantly reduced, which results in large computational gains. The algorithm is tested on various types of wrapped phase imagery, reporting speedup factors of up to 500. The source code of the algorithm is publicly available.

Video-rate processing in tomographic phase microscopy of biological cells using CUDA

We suggest a new implementation for rapid reconstruction of three-dimensional (3-D) refractive index (RI) maps of biological cells acquired by tomographic phase microscopy (TPM). The TPM computational reconstruction process is extremely time consuming, making the analysis of large data sets unreasonably slow and the real-time 3-D visualization of the results impossible. Our implementation uses new phase extraction, phase unwrapping and Fourier slice algorithms, suitable for efficient CPU or GPU implementations. The experimental setup includes an external off-axis interferometric module connected to an inverted microscope illuminated coherently. We used single cell rotation by micro-manipulation to obtain interferometric projections from 73 viewing angles over a 180° angular range. Our parallel algorithms were implemented using Nvidia's CUDA C platform, running on Nvidia's Tesla K20c GPU. This implementation yields, for the first time to our knowledge, a 3-D reconstruction rate higher than video rate of 25 frames per second for 256 × 256-pixel interferograms with 73 different projection angles (64 × 64 × 64 output). This allows us to calculate additional cellular parameters, while still processing faster than video rate. This technique is expected to find uses for real-time 3-D cell visualization and processing, while yielding fast feedback for medical diagnosis and cell sorting.