Speeding up reconstruction of 3D tomograms in holographic flow cytometry via deep learning

Neural network-based processing and reconstruction of compromised biophotonic image data

In recent years, the integration of deep learning techniques with biophotonic setups has opened new horizons in bioimaging. A compelling trend in this field involves deliberately compromising certain measurement metrics to engineer better bioimaging tools in terms of e.g., cost, speed, and form-factor, followed by compensating for the resulting defects through the utilization of deep learning models trained on a large amount of ideal, superior or alternative data. This strategic approach has found increasing popularity due to its potential to enhance various aspects of biophotonic imaging. One of the primary motivations for employing this strategy is the pursuit of higher temporal resolution or increased imaging speed, critical for capturing fine dynamic biological processes. Additionally, this approach offers the prospect of simplifying hardware requirements and complexities, thereby making advanced imaging standards more accessible in terms of cost and/or size. This article provides an in-depth review of the diverse measurement aspects that researchers intentionally impair in their biophotonic setups, including the point spread function (PSF), signal-to-noise ratio (SNR), sampling density, and pixel resolution. By deliberately compromising these metrics, researchers aim to not only recuperate them through the application of deep learning networks, but also bolster in return other crucial parameters, such as the field of view (FOV), depth of field (DOF), and space-bandwidth product (SBP). Throughout this article, we discuss various biophotonic methods that have successfully employed this strategic approach. These techniques span a wide range of applications and showcase the versatility and effectiveness of deep learning in the context of compromised biophotonic data. Finally, by offering our perspectives on the exciting future possibilities of this rapidly evolving concept, we hope to motivate our readers from various disciplines to explore novel ways of balancing hardware compromises with compensation via artificial intelligence (AI).

Rapid flowing cells localization enabled by spatiotemporal manipulation of their holographic patterns

Lab-on-a-Chip microfluidic devices present an innovative and cost-effective platform in the current trend of miniaturization and simplification of imaging flow cytometry; they are excellent candidates for high-throughput single-cell analysis. In such microfluidic platforms, cell tracking becomes a fundamental tool for investigating biophysical processes, from intracellular dynamics to the characterization of cell motility and migration. However, high-throughput and long-term cell tracking puts a high demand on the consumption of computing resources. Here, we propose a novel strategy to achieve rapid 3D cell localizations along the microfluidic channel. This method is based on the spatiotemporal manipulation of recorded holographic interference fringes, and it allows fast and precise localization of cells without performing complete holographic reconstruction. Conventional holographic tracking is typically based on the phase contrast obtained by decoupling the calculation of optical axial and transverse coordinates. Computing time and resource consumption may increase because all the frames need to be calculated in the Fourier domain. In our proposed method, the 2D transverse positions are directly located by morphological calculation based on the hologram. The complex-amplitude wavefronts are directly reconstructed by spatiotemporal phase shifting to calculate the axial position by the refocusing criterion. Only spatial calculation is considered in the proposed method. We demonstrate that the computational time of transverse tracking is only one-tenth of the conventional method, while the total computational time of the proposed method decreases up to 54% with respect to the conventional approach. The proposed approach can open the route for analyzing flow cytometry in quantitative phase microscopy assays.

Investigation on lysosomal accumulation by a quantitative analysis of 2D phase-maps in digital holography microscopy

Lysosomes are the terminal end of catabolic pathways in the cell, as well as signaling centers performing important functions such as the recycling of macromolecules, organelles, and nutrient adaptation. The importance of lysosomes in human health is supported by the fact that the deficiency of most lysosomal genes causes monogenic diseases called as a group Lysosomal Storage Diseases (LSDs). A common phenotypic hallmark of LSDs is the expansion of the lysosomal compartment that can be detected by using conventional imaging methods based on immunofluorescence protocols or overexpression of tagged lysosomal proteins. These methods require the alteration of the cellular architecture (i.e., due to fixation methods), can alter the behavior of cells (i.e., by the overexpression of proteins), and require sample preparation and the accurate selection of compatible fluorescent markers in relation to the type of analysis, therefore limiting the possibility of characterizing cellular status with simplicity. Therefore, a quantitative and label-free methodology, such as Quantitative Phase Imaging through Digital Holographic (QPI-DH), for the microscopic imaging of lysosomes in health and disease conditions may represent an important advance to study and effectively diagnose the presence of lysosomal storage in human disease. Here we proof the effectiveness of the QPI-DH method in accomplishing the detection of the lysosomal compartment using mouse embryonic fibroblasts (MEFs) derived from a Mucopolysaccharidosis type III-A (MSP-IIIA) mouse model, and comparing them with wild-type (WT) MEFs. We found that it is possible to identify label-free biomarkers able to supply a first pre-screening of the two populations, thus showing that QPI-DH can be a suitable candidate to surpass fluorescent drawbacks in the detection of lysosomes dysfunction. An appropriate numerical procedure was developed for detecting and evaluate such cellular substructures from in vitro cells cultures. Results reported in this study are encouraging about the further development of the proposed QPI-DH approach for such type of investigations about LSDs.

Label-free cell classification in holographic flow cytometry through an unbiased learning strategy

Nowadays, label-free imaging flow cytometry at the single-cell level is considered the stepforward lab-on-a-chip technology to address challenges in clinical diagnostics, biology, life sciences and healthcare. In this framework, digital holography in microscopy promises to be a powerful imaging modality thanks to its multi-refocusing and label-free quantitative phase imaging capabilities, along with the encoding of the highest information content within the imaged samples. Moreover, the recent achievements of new data analysis tools for cell classification based on deep/machine learning, combined with holographic imaging, are urging these systems toward the effective implementation of point of care devices. However, the generalization capabilities of learning-based models may be limited from biases caused by data obtained from other holographic imaging settings and/or different processing approaches. In this paper, we propose a combination of a Mask R-CNN to detect the cells, a convolutional auto-encoder, used to the image feature extraction and operating on unlabelled data, thus overcoming the bias due to data coming from different experimental settings, and a feedforward neural network for single cell classification, that operates on the above extracted features. We demonstrate the proposed approach in the challenging classification task related to the identification of drug-resistant endometrial cancer cells.

Beyond fluorescence: advances in computational label-free full specificity in 3D quantitative phase microscopy

Despite remarkable progresses in quantitative phase imaging (QPI) microscopes, their wide acceptance is limited due to the lack of specificity compared with the well-established fluorescence microscopy. In fact, the absence of fluorescent tag prevents to identify subcellular structures in single cells, making challenging the interpretation of label-free 2D and 3D phase-contrast data. Great effort has been made by many groups worldwide to address and overcome such limitation. Different computational methods have been proposed and many more are currently under investigation to achieve label-free microscopic imaging at single-cell level to recognize and quantify different subcellular compartments. This route promises to bridge the gap between QPI and FM for real-world applications.

AI-driven projection tomography with multicore fibre-optic cell rotation

Optical tomography has emerged as a non-invasive imaging method, providing three-dimensional insights into subcellular structures and thereby enabling a deeper understanding of cellular functions, interactions, and processes. Conventional optical tomography methods are constrained by a limited illumination scanning range, leading to anisotropic resolution and incomplete imaging of cellular structures. To overcome this problem, we employ a compact multi-core fibre-optic cell rotator system that facilitates precise optical manipulation of cells within a microfluidic chip, achieving full-angle projection tomography with isotropic resolution. Moreover, we demonstrate an AI-driven tomographic reconstruction workflow, which can be a paradigm shift from conventional computational methods, often demanding manual processing, to a fully autonomous process. The performance of the proposed cell rotation tomography approach is validated through the three-dimensional reconstruction of cell phantoms and HL60 human cancer cells. The versatility of this learning-based tomographic reconstruction workflow paves the way for its broad application across diverse tomographic imaging modalities, including but not limited to flow cytometry tomography and acoustic rotation tomography. Therefore, this AI-driven approach can propel advancements in cell biology, aiding in the inception of pioneering therapeutics, and augmenting early-stage cancer diagnostics.

On the use of deep learning for phase recovery

Phase recovery (PR) refers to calculating the phase of the light field from its intensity measurements. As exemplified from quantitative phase imaging and coherent diffraction imaging to adaptive optics, PR is essential for reconstructing the refractive index distribution or topography of an object and correcting the aberration of an imaging system. In recent years, deep learning (DL), often implemented through deep neural networks, has provided unprecedented support for computational imaging, leading to more efficient solutions for various PR problems. In this review, we first briefly introduce conventional methods for PR. Then, we review how DL provides support for PR from the following three stages, namely, pre-processing, in-processing, and post-processing. We also review how DL is used in phase image processing. Finally, we summarize the work in DL for PR and provide an outlook on how to better use DL to improve the reliability and efficiency of PR. Furthermore, we present a live-updating resource ( https://github.com/kqwang/phase-recovery ) for readers to learn more about PR.

In-focus quantitative phase imaging from defocused off-axis holograms: synergistic reconstruction framework

Digital holographic microscopy (DHM) enables the three-dimensional (3D) reconstruction of quantitative phase distributions from a defocused hologram. Traditional computational algorithms follow a sequential approach in which one first reconstructs the complex amplitude distribution and later applies focusing algorithms to provide an in-focus phase map. In this work, we have developed a synergistic computational framework to compensate for the linear tilt introduced in off-axis DHM systems and autofocus the defocused holograms by minimizing a cost function, providing in-focus reconstructed phase images without phase distortions. The proposed computational tool has been validated in defocused holograms of human red blood cells and three-dimensional images of dynamic sperm cells.

Artificial intelligence-enabled quantitative phase imaging methods for life sciences

Quantitative phase imaging, integrated with artificial intelligence, allows for the rapid and label-free investigation of the physiology and pathology of biological systems. This review presents the principles of various two-dimensional and three-dimensional label-free phase imaging techniques that exploit refractive index as an intrinsic optical imaging contrast. In particular, we discuss artificial intelligence-based analysis methodologies for biomedical studies including image enhancement, segmentation of cellular or subcellular structures, classification of types of biological samples and image translation to furnish subcellular and histochemical information from label-free phase images. We also discuss the advantages and challenges of artificial intelligence-enabled quantitative phase imaging analyses, summarize recent notable applications in the life sciences, and cover the potential of this field for basic and industrial research in the life sciences.

Phenotyping neuroblastoma cells through intelligent scrutiny of stain-free biomarkers in holographic flow cytometry

To efficiently tackle certain tumor types, finding new biomarkers for rapid and complete phenotyping of cancer cells is highly demanded. This is especially the case for the most common pediatric solid tumor of the sympathetic nervous system, namely, neuroblastoma (NB). Liquid biopsy is in principle a very promising tool for this purpose, but usually enrichment and isolation of circulating tumor cells in such patients remain difficult due to the unavailability of universal NB cell-specific surface markers. Here, we show that rapid screening and phenotyping of NB cells through stain-free biomarkers supported by artificial intelligence is a viable route for liquid biopsy. We demonstrate the concept through a flow cytometry based on label-free holographic quantitative phase-contrast microscopy empowered by machine learning. In detail, we exploit a hierarchical decision scheme where at first level NB cells are classified from monocytes with 97.9% accuracy. Then we demonstrate that different phenotypes are discriminated within NB class. Indeed, for each cell classified as NB its belonging to one of four NB sub-populations (i.e., CHP212, SKNBE2, SHSY5Y, and SKNSH) is evaluated thus achieving accuracy in the range 73.6%-89.1%. The achieved results solve the realistic problem related to the identification circulating tumor cell, i.e., the possibility to recognize and detect tumor cells morphologically similar to blood cells, which is the core issue in liquid biopsy based on stain-free microscopy. The presented approach operates at lab-on-chip scale and emulates real-world scenarios, thus representing a future route for liquid biopsy by exploiting intelligent biomedical imaging.

Enhancing single-cell biology through advanced AI-powered microfluidics

Microfluidic technology has largely benefited both fundamental biological research and translational clinical diagnosis with its advantages in high-throughput, single-cell resolution, high integrity, and wide-accessibility. Despite the merits we obtained from microfluidics in the last two decades, the current requirement of intelligence in biomedicine urges the microfluidic technology to process biological big data more efficiently and intelligently. Thus, the current readout technology based on the direct detection of the signals in either optics or electrics was not able to meet the requirement. The implementation of artificial intelligence (AI) in microfluidic technology matches up with the large-scale data usually obtained in the high-throughput assays of microfluidics. At the same time, AI is able to process the multimodal datasets obtained from versatile microfluidic devices, including images, videos, electric signals, and sequences. Moreover, AI provides the microfluidic technology with the capability to understand and decipher the obtained datasets rather than simply obtaining, which eventually facilitates fundamental and translational research in many areas, including cell type discovery, cell signaling, single-cell genetics, and diagnosis. In this Perspective, we will highlight the recent advances in employing AI for single-cell biology and present an outlook on the future direction with more advanced AI algorithms.

Holographic flow scanning cytometry overcomes depth of focus limits and smartly adapts to microfluidic speed

Space-time digital holography (STDH) maps holograms in a hybrid space-time domain to achieve extended field of view, resolution enhanced, quantitative phase-contrast microscopy and velocimetry of flowing objects in a label-free modality. In STDH, area sensors can be replaced by compact and faster linear sensor arrays to augment the imaging throughput and to compress data from a microfluidic video sequence into one single hybrid hologram. However, in order to ensure proper imaging, the velocity of the objects in microfluidic channels has to be well-matched to the acquisition frame rate, which is the major constraint of the method. Also, imaging all the flowing samples in focus at the same time, while avoiding hydrodynamic focusing devices, is a highly desirable goal. Here we demonstrate a novel processing pipeline that addresses non-ideal flow conditions and is capable of returning the correct and extended focus phase contrast mapping of an entire microfluidic experiment in a single image. We apply this novel processing strategy to recover phase imaging of flowing HeLa cells in a lab-on-a-chip platform even when severely undersampled due to too fast flow while ensuring that all cells are in focus.

Synchronous Phase-Shifting Interference for High Precision Phase Imaging of Objects Using Common Optics

Quantitative phase imaging and measurement of surface topography and fluid dynamics for objects, especially for moving objects, is critical in various fields. Although effective, existing synchronous phase-shifting methods may introduce additional phase changes in the light field due to differences in optical paths or need specific optics to implement synchronous phase-shifting, such as the beamsplitter with additional anti-reflective coating and a micro-polarizer array. Therefore, we propose a synchronous phase-shifting method based on the Mach-Zehnder interferometer to tackle these issues in existing methods. The proposed method uses common optics to simultaneously acquire four phase-shifted digital holograms with equal optical paths for object and reference waves. Therefore, it can be used to reconstruct the phase distribution of static and dynamic objects with high precision and high resolution. In the experiment, the theoretical resolution of the proposed system was 1.064 µm while the actual resolution could achieve 1.381 µm, which was confirmed by measuring a phase-only resolution chart. Besides, the dynamic phase imaging of a moving standard object was completed to verify the proposed system's effectiveness. The experimental results show that our proposed method is suitable and promising in dynamic phase imaging and measurement of moving objects using phase-shifting digital holography.

Label-free liquid biopsy through the identification of tumor cells by machine learning-powered tomographic phase imaging flow cytometry

Image-based identification of circulating tumor cells in microfluidic cytometry condition is one of the most challenging perspectives in the Liquid Biopsy scenario. Here we show a machine learning-powered tomographic phase imaging flow cytometry system capable to provide high-throughput 3D phase-contrast tomograms of each single cell. In fact, we show that discrimination of tumor cells against white blood cells is potentially achievable with the aid of artificial intelligence in a label-free flow-cyto-tomography method. We propose a hierarchical machine learning decision-maker, working on a set of features calculated from the 3D tomograms of the cells' refractive index. We prove that 3D morphological features are adequately distinctive to identify tumor cells versus the white blood cell background in the first stage and, moreover, in recognizing the tumor type at the second decision step. Proof-of-concept experiments are shown, in which two different tumor cell lines, namely neuroblastoma cancer cells and ovarian cancer cells, are used against monocytes. The reported results allow claiming the identification of tumor cells with a success rate higher than 97% and with an accuracy over 97% in discriminating between the two cancer cell types, thus opening in a near future the route to a new Liquid Biopsy tool for detecting and classifying circulating tumor cells in blood by stain-free method.

Microsystem Advances through Integration with Artificial Intelligence

Microfluidics is a rapidly growing discipline that involves studying and manipulating fluids at reduced length scale and volume, typically on the scale of micro- or nanoliters. Under the reduced length scale and larger surface-to-volume ratio, advantages of low reagent consumption, faster reaction kinetics, and more compact systems are evident in microfluidics. However, miniaturization of microfluidic chips and systems introduces challenges of stricter tolerances in designing and controlling them for interdisciplinary applications. Recent advances in artificial intelligence (AI) have brought innovation to microfluidics from design, simulation, automation, and optimization to bioanalysis and data analytics. In microfluidics, the Navier-Stokes equations, which are partial differential equations describing viscous fluid motion that in complete form are known to not have a general analytical solution, can be simplified and have fair performance through numerical approximation due to low inertia and laminar flow. Approximation using neural networks trained by rules of physical knowledge introduces a new possibility to predict the physicochemical nature. The combination of microfluidics and automation can produce large amounts of data, where features and patterns that are difficult to discern by a human can be extracted by machine learning. Therefore, integration with AI introduces the potential to revolutionize the microfluidic workflow by enabling the precision control and automation of data analysis. Deployment of smart microfluidics may be tremendously beneficial in various applications in the future, including high-throughput drug discovery, rapid point-of-care-testing (POCT), and personalized medicine. In this review, we summarize key microfluidic advances integrated with AI and discuss the outlook and possibilities of combining AI and microfluidics.

Digital holographic approaches to the detection and characterization of microplastics in water environments

Microplastic (MP) pollution is seriously threatening the environmental health of the world, which has accelerated the development of new identification and characterization methods. Digital holography (DH) is one of the emerging tools to detect MPs in a high-throughput flow. Here, we review advances in MP screening by DH. We examine the problem from both the hardware and software viewpoints. Automatic analysis based on smart DH processing is reported by highlighting the role played by artificial intelligence for classification and regression tasks. In this framework, the continuous development and availability in recent years of field-portable holographic flow cytometers for water monitoring also is discussed.

Non-Invasive and Minute-Frequency 3D Tomographic Imaging Enabling Long-Term Spatiotemporal Observation of Single Cell Fate

Non-invasive and rapid imaging technique at subcellular resolution is significantly important for multiple biological applications such as cell fate study. Label-free refractive-index (RI)-based 3D tomographic imaging constitutes an excellent candidate for 3D imaging of cellular structures, but its full potential in long-term spatiotemporal cell fate observation is locked due to the lack of an efficient integrated system. Here, a long-term 3D RI imaging system incorporating a cutting-edge white light diffraction phase microscopy module with spatiotemporal stability, and an acoustofluidic device to roll and culture single cells in a customized live cell culture chamber is reported. Using this system, 3D RI imaging experiments are conducted for 250 cells and demonstrate efficient cell identification with high accuracy. Importantly, long-term and frequency-on-demand 3D RI imaging of K562 and MCF-7 cancer cells reveal different characteristics during normal cell growth, drug-induced cell apoptosis, and necrosis of drug-treated cells. Overall, it is believed that the proposed 3D tomographic imaging technique opens up a new avenue for visualizing intracellular structures and will find many applications such as disease diagnosis and nanomedicine.

On monocytes and lymphocytes biolens clustering by in flow holographic microscopy

Live cells act as biological lenses and can be employed as real-world optical components in bio-hybrid systems. Imaging at nanoscale, optical tweezers, lithography and also photonic waveguiding are some of the already proven functionalities, boosted by the advantage that cells are fully biocompatible for intra-body applications. So far, various cell types have been studied for this purpose, such as red blood cells, bacterial cells, stem cells and yeast cells. White Blood Cells (WBCs) play a very important role in the regulation of the human body activities and are usually monitored for assessing its health. WBCs can be considered bio-lenses but, to the best of our knowledge, characterization of their optical properties have not been investigated yet. Here, we report for the first time an accurate study of two model classes of WBCs (i.e., monocytes and lymphocytes) by means of a digital holographic microscope coupled with a microfluidic system, assuming WBCs bio-lens characteristics. Thus, quantitative phase maps for many WBCs have been retrieved in flow-cytometry (FC) by achieving a significant statistical analysis to prove the enhancement in differentiation among sphere-like bio-lenses according to their sizes (i.e., diameter d) exploiting intensity parameters of the modulated light in proximity of the cell optical axis. We show that the measure of the low intensity area (S: I z < I th z ) in a fixed plane, is a feasible parameter for cell clustering, while achieving robustness against experimental misalignments and allowing to adjust the measurement sensitivity in post-processing. 2D scatterplots of the identified parameters (d-S) show better differentiation respect to the 1D case. The results show that the optical focusing properties of WBCs allow the clustering of the two populations by means of a mere morphological analysis, thus leading to the new concept of cell-optical-fingerprint avoiding fluorescent dyes. This perspective can open new routes in biomedical sciences, such as the chance to find optical-biomarkers at single cell level for label-free diagnosis.

Stain-free identification of cell nuclei using tomographic phase microscopy in flow cytometry

Quantitative Phase Imaging (QPI) has gained popularity in bioimaging because it can avoid the need for cell staining, which in some cases is difficult or impossible. However, as a result, QPI does not provide labelling of various specific intracellular structures. Here we show a novel computational segmentation method based on statistical inference that makes it possible for QPI techniques to identify the cell nucleus. We demonstrate the approach with refractive index tomograms of stain-free cells reconstructed through the tomographic phase microscopy in flow cytometry mode. In particular, by means of numerical simulations and two cancer cell lines, we demonstrate that the nucleus can be accurately distinguished within the stain-free tomograms. We show that our experimental results are consistent with confocal fluorescence microscopy (FM) data and microfluidic cytofluorimeter outputs. This is a significant step towards extracting specific three-dimensional intracellular structures directly from the phase-contrast data in a typical flow cytometry configuration.

Finding intracellular lipid droplets from the single-cell biolens' signature in a holographic flow-cytometry assay

In recent years, intracellular LDs have been discovered to play an important role in several pathologies. Therefore, detection of LDs would provide an in-demand diagnostic tool if coupled with flow-cytometry to give significant statistical analysis and especially if the diagnosis is made in full non-invasive mode. Here we combine the experimental results of in-flow tomographic phase microscopy with a suited numerical simulation to demonstrate that intracellular LDs can be easily detected through a label-free approach based on the direct analysis of the 2D quantitative phase maps recorded by a holographic flow cytometer. In fact, we demonstrate that the presence of LDs affects the optical focusing lensing features of the embracing cell, which can be considered a biological lens. The research was conducted on white blood cells (i.e., lymphocytes and monocytes) and ovarian cancer cells. Results show that the biolens properties of cells can be a rapid biomarker that aids in boosting the diagnosis of LDs-related pathologies by means of the holographic flow-cytometry assay for fast, non-destructive, and high-throughput screening of statistically significant number of cells.

Developing a Reliable Holographic Flow Cyto-Tomography Apparatus by Optimizing the Experimental Layout and Computational Processing

Digital Holographic Tomography (DHT) has recently been established as a means of retrieving the 3D refractive index mapping of single cells. To make DHT a viable system, it is necessary to develop a reliable and robust holographic apparatus in order that such technology can be utilized outside of specialized optics laboratories and operated in the in-flow modality. In this paper, we propose a quasi-common-path lateral-shearing holographic optical set-up to be used, for the first time, for DHT in a flow-cytometer modality. The proposed solution is able to withstand environmental vibrations that can severely affect the interference process. Furthermore, we have scaled down the system while ensuring that a full 360° rotation of the cells occurs in the field-of-view, in order to retrieve 3D phase-contrast tomograms of single cells flowing along a microfluidic channel. This was achieved by setting the camera sensor at 45° with respect to the microfluidic direction. Additional optimizations were made to the computational elements to ensure the reliable retrieval of 3D refractive index distributions by demonstrating an effective method of tomographic reconstruction, based on high-order total variation. The results were first demonstrated using realistic 3D numerical phantom cells to assess the performance of the proposed high-order total variation method in comparison with the gold-standard algorithm for tomographic reconstructions: namely, filtered back projection. Then, the proposed DHT system and the processing pipeline were experimentally validated for monocytes and mouse embryonic fibroblast NIH-3T3 cells lines. Moreover, the repeatability of these tomographic measurements was also investigated by recording the same cell multiple times and quantifying the ability to provide reliable and comparable tomographic reconstructions, as confirmed by a correlation coefficient greater than 95%. The reported results represent various steps forward in several key aspects of in-flow DHT, thus paving the way for its use in real-world applications.

Fourier Imager Network (FIN): A deep neural network for hologram reconstruction with superior external generalization

Deep learning-based image reconstruction methods have achieved remarkable success in phase recovery and holographic imaging. However, the generalization of their image reconstruction performance to new types of samples never seen by the network remains a challenge. Here we introduce a deep learning framework, termed Fourier Imager Network (FIN), that can perform end-to-end phase recovery and image reconstruction from raw holograms of new types of samples, exhibiting unprecedented success in external generalization. FIN architecture is based on spatial Fourier transform modules that process the spatial frequencies of its inputs using learnable filters and a global receptive field. Compared with existing convolutional deep neural networks used for hologram reconstruction, FIN exhibits superior generalization to new types of samples, while also being much faster in its image inference speed, completing the hologram reconstruction task in ~0.04 s per 1 mm² of the sample area. We experimentally validated the performance of FIN by training it using human lung tissue samples and blindly testing it on human prostate, salivary gland tissue and Pap smear samples, proving its superior external generalization and image reconstruction speed. Beyond holographic microscopy and quantitative phase imaging, FIN and the underlying neural network architecture might open up various new opportunities to design broadly generalizable deep learning models in computational imaging and machine vision fields.

Efficient Computer-Generated Holography Based on Mixed Linear Convolutional Neural Networks

Imaging based on computer-generated holography using traditional methods has the problems of poor quality and long calculation cycles. However, recently, the development of deep learning has provided new ideas for this problem. Here, an efficient computer-generated holography (ECGH) method is proposed for computational holographic imaging. This method can be used for computational holographic imaging based on mixed linear convolutional neural networks (MLCNN). By introducing fully connected layers in the network, the suggested design is more powerful and efficient at information mining and information exchange. Using the ECGH, the pure phase image required can be obtained after calculating the custom light field. Compared with traditional computed holography based on deep learning, the method used here can reduce the number of network parameters needed for network training by about two-thirds while obtaining a high-quality image in the reconstruction, and the network structure has the potential to solve various image-reconstruction problems.