Intelligent frequency-shifted optofluidic time-stretch quantitative phase imaging

Classification and counting of cells in brightfield microscopy images: an application of convolutional neural networks

Microscopy is integral to medical research, facilitating the exploration of various biological questions, notably cell quantification. However, this process's time-consuming and error-prone nature, attributed to human intervention or automated methods usually applied to fluorescent images, presents challenges. In response, machine learning algorithms have been integrated into microscopy, automating tasks and constructing predictive models from vast datasets. These models adeptly learn representations for object detection, image segmentation, and target classification. An advantageous strategy involves utilizing unstained images, preserving cell integrity and enabling morphology-based classification-something hindered when fluorescent markers are used. The aim is to introduce a model proficient in classifying distinct cell lineages in digital contrast microscopy images. Additionally, the goal is to create a predictive model identifying lineage and determining optimal quantification of cell numbers. Employing a CNN machine learning algorithm, a classification model predicting cellular lineage achieved a remarkable accuracy of 93%, with ROC curve results nearing 1.0, showcasing robust performance. However, some lineages, namely SH-SY5Y (78%), HUH7_mayv (85%), and A549 (88%), exhibited slightly lower accuracies. These outcomes not only underscore the model's quality but also emphasize CNNs' potential in addressing the inherent complexities of microscopic images.

Interpretable unsupervised learning enables accurate clustering with high-throughput imaging flow cytometry

A primary challenge of high-throughput imaging flow cytometry (IFC) is to analyze the vast amount of imaging data, especially in applications where ground truth labels are unavailable or hard to obtain. We present an unsupervised deep embedding algorithm, the Deep Convolutional Autoencoder-based Clustering (DCAEC) model, to cluster label-free IFC images without any prior knowledge of input labels. The DCAEC model first encodes the input images into the latent representations and then clusters based on the latent representations. Using the DCAEC model, we achieve a balanced accuracy of 91.9% for human white blood cell (WBC) clustering and 97.9% for WBC/leukemia clustering using the 3D IFC images and 3D DCAEC model. Above all, although no human recognizable features can separate the clusters of cells with protein localization, we demonstrate the fused DCAEC model can achieve a cluster balanced accuracy of 85.3% from the label-free 2D transmission and 3D side scattering images. To reveal how the neural network recognizes features beyond human ability, we use the gradient-weighted class activation mapping method to discover the cluster-specific visual patterns automatically. Evaluation results show that the automatically identified salient image regions have strong cluster-specific visual patterns for different clusters, which we believe is a stride for the interpretable neural network for cell analysis with high-throughput IFCs.

Microsystem Advances through Integration with Artificial Intelligence

Microfluidics is a rapidly growing discipline that involves studying and manipulating fluids at reduced length scale and volume, typically on the scale of micro- or nanoliters. Under the reduced length scale and larger surface-to-volume ratio, advantages of low reagent consumption, faster reaction kinetics, and more compact systems are evident in microfluidics. However, miniaturization of microfluidic chips and systems introduces challenges of stricter tolerances in designing and controlling them for interdisciplinary applications. Recent advances in artificial intelligence (AI) have brought innovation to microfluidics from design, simulation, automation, and optimization to bioanalysis and data analytics. In microfluidics, the Navier-Stokes equations, which are partial differential equations describing viscous fluid motion that in complete form are known to not have a general analytical solution, can be simplified and have fair performance through numerical approximation due to low inertia and laminar flow. Approximation using neural networks trained by rules of physical knowledge introduces a new possibility to predict the physicochemical nature. The combination of microfluidics and automation can produce large amounts of data, where features and patterns that are difficult to discern by a human can be extracted by machine learning. Therefore, integration with AI introduces the potential to revolutionize the microfluidic workflow by enabling the precision control and automation of data analysis. Deployment of smart microfluidics may be tremendously beneficial in various applications in the future, including high-throughput drug discovery, rapid point-of-care-testing (POCT), and personalized medicine. In this review, we summarize key microfluidic advances integrated with AI and discuss the outlook and possibilities of combining AI and microfluidics.

Optofluidic imaging meets deep learning: from merging to emerging

Propelled by the striking advances in optical microscopy and deep learning (DL), the role of imaging in lab-on-a-chip has dramatically been transformed from a silo inspection tool to a quantitative "smart" engine. A suite of advanced optical microscopes now enables imaging over a range of spatial scales (from molecules to organisms) and temporal window (from microseconds to hours). On the other hand, the staggering diversity of DL algorithms has revolutionized image processing and analysis at the scale and complexity that were once inconceivable. Recognizing these exciting but overwhelming developments, we provide a timely review of their latest trends in the context of lab-on-a-chip imaging, or coined optofluidic imaging. More importantly, here we discuss the strengths and caveats of how to adopt, reinvent, and integrate these imaging techniques and DL algorithms in order to tailor different lab-on-a-chip applications. In particular, we highlight three areas where the latest advances in lab-on-a-chip imaging and DL can form unique synergisms: image formation, image analytics and intelligent image-guided autonomous lab-on-a-chip. Despite the on-going challenges, we anticipate that they will represent the next frontiers in lab-on-a-chip imaging that will spearhead new capabilities in advancing analytical chemistry research, accelerating biological discovery, and empowering new intelligent clinical applications.

Studying the efficacy of antiplatelet drugs on atherosclerosis by optofluidic imaging on a chip

Vascular stenosis caused by atherosclerosis instigates activation and aggregation of platelets, eventually resulting in thrombus formation. Although antiplatelet drugs are commonly used to inhibit platelet activation and aggregation, they unfortunately cannot prevent recurrent thrombotic events in patients with atherosclerosis. This is partially due to the limited understanding of the efficacy of antiplatelet drugs in the complex hemodynamic environment of vascular stenosis. Conventional methods for evaluating the efficacy of antiplatelet drugs under stenosis either fail to simulate the hemodynamic environment of vascular stenosis characterized by high shear stress and recirculatory flow or lack spatial resolution in their analytical techniques to statistically identify and characterize platelet aggregates. Here we propose and experimentally demonstrate a method comprising an in vitro 3D stenosis microfluidic chip and an optical time-stretch quantitative phase imaging system for studying the efficacy of antiplatelet drugs under stenosis. Our method simulates the atherogenic flow environment of vascular stenosis while enabling high-resolution and statistical analysis of platelet aggregates. Using our method, we distinguished the efficacy of three antiplatelet drugs, acetylsalicylic acid (ASA), cangrelor, and eptifibatide, for inhibiting platelet aggregation induced by stenosis. Specifically, ASA failed to inhibit stenosis-induced platelet aggregation, while eptifibatide and cangrelor showed high and moderate efficacy, respectively. Furthermore, we demonstrated that the drugs tested also differed in their efficacy for inhibiting platelet aggregation synergistically induced by stenosis and agonists (e.g., adenosine diphosphate, and collagen). Taken together, our method is an effective tool for investigating the efficacy of antiplatelet drugs under vascular stenosis, which could assist the development of optimal pharmacologic strategies for patients with atherosclerosis.

Finding intracellular lipid droplets from the single-cell biolens' signature in a holographic flow-cytometry assay

In recent years, intracellular LDs have been discovered to play an important role in several pathologies. Therefore, detection of LDs would provide an in-demand diagnostic tool if coupled with flow-cytometry to give significant statistical analysis and especially if the diagnosis is made in full non-invasive mode. Here we combine the experimental results of in-flow tomographic phase microscopy with a suited numerical simulation to demonstrate that intracellular LDs can be easily detected through a label-free approach based on the direct analysis of the 2D quantitative phase maps recorded by a holographic flow cytometer. In fact, we demonstrate that the presence of LDs affects the optical focusing lensing features of the embracing cell, which can be considered a biological lens. The research was conducted on white blood cells (i.e., lymphocytes and monocytes) and ovarian cancer cells. Results show that the biolens properties of cells can be a rapid biomarker that aids in boosting the diagnosis of LDs-related pathologies by means of the holographic flow-cytometry assay for fast, non-destructive, and high-throughput screening of statistically significant number of cells.

Automatic whole blood cell analysis from blood smear using label-free multi-modal imaging with deep neural networks

Whole blood cell analysis is widely used in medical applications since its results are indicators for diagnosing a series of diseases. In this work, we report automatic whole blood cell analysis from blood smear using label-free multi-modal imaging with deep neural networks. First, a commercial microscope equipped with our developed Phase Real-time Microscope Camera (PhaseRMiC) obtains both bright-field and quantitative phase images. Then, these images are automatically processed by our designed blood smear recognition networks (BSRNet) that recognize erythrocytes, leukocytes and platelets. Finally, blood cell parameters such as counts, shapes and volumes can be extracted according to both quantitative phase images and automatic recognition results. The proposed whole blood cell analysis technique provides high-quality blood cell images and supports accurate blood cell recognition and analysis. Moreover, this approach requires rather simple and cost-effective setups as well as easy and rapid sample preparations. Therefore, this proposed method has great potential application in blood testing aiming at disease diagnostics.

A Review of Optical Neural Networks

With the continuous miniaturization of conventional integrated circuits, obstacles such as excessive cost, increased resistance to electronic motion, and increased energy consumption are gradually slowing down the development of electrical computing and constraining the application of deep learning. Optical neuromorphic computing presents various opportunities and challenges compared with the realm of electronics. Algorithms running on optical hardware have the potential to meet the growing computational demands of deep learning and artificial intelligence. Here, we review the development of optical neural networks and compare various research proposals. We focus on fiber-based neural networks. Finally, we describe some new research directions and challenges.

Deep imaging flow cytometry

Imaging flow cytometry (IFC) has become a powerful tool for diverse biomedical applications by virtue of its ability to image single cells in a high-throughput manner. However, there remains a challenge posed by the fundamental trade-off between throughput, sensitivity, and spatial resolution. Here we present deep-learning-enhanced imaging flow cytometry (dIFC) that circumvents this trade-off by implementing an image restoration algorithm on a virtual-freezing fluorescence imaging (VIFFI) flow cytometry platform, enabling higher throughput without sacrificing sensitivity and spatial resolution. A key component of dIFC is a high-resolution (HR) image generator that synthesizes "virtual" HR images from the corresponding low-resolution (LR) images acquired with a low-magnification lens (10×/0.4-NA). For IFC, a low-magnification lens is favorable because of reduced image blur of cells flowing at a higher speed, which allows higher throughput. We trained and developed the HR image generator with an architecture containing two generative adversarial networks (GANs). Furthermore, we developed dIFC as a method by combining the trained generator and IFC. We characterized dIFC using Chlamydomonas reinhardtii cell images, fluorescence in situ hybridization (FISH) images of Jurkat cells, and Saccharomyces cerevisiae (budding yeast) cell images, showing high similarities of dIFC images to images obtained with a high-magnification lens (40×/0.95-NA), at a high flow speed of 2 m s^-1. We lastly employed dIFC to show enhancements in the accuracy of FISH-spot counting and neck-width measurement of budding yeast cells. These results pave the way for statistical analysis of cells with high-dimensional spatial information.

Artificial intelligence-powered microfluidics for nanomedicine and materials synthesis

Artificial intelligence (AI) is an emerging technology with great potential, and its robust calculation and analysis capabilities are unmatched by traditional calculation tools. With the promotion of deep learning and open-source platforms, the threshold of AI has also become lower. Combining artificial intelligence with traditional fields to create new fields of high research and application value has become a trend. AI has been involved in many disciplines, such as medicine, materials, energy, and economics. The development of AI requires the support of many kinds of data, and microfluidic systems can often mine object data on a large scale to support AI. Due to the excellent synergy between the two technologies, excellent research results have emerged in many fields. In this review, we briefly review AI and microfluidics and introduce some applications of their combination, mainly in nanomedicine and material synthesis. Finally, we discuss the development trend of the combination of the two technologies.

Label-Free White Blood Cell Classification Using Refractive Index Tomography and Deep Learning

Objective and Impact Statement. We propose a rapid and accurate blood cell identification method exploiting deep learning and label-free refractive index (RI) tomography. Our computational approach that fully utilizes tomographic information of bone marrow (BM) white blood cell (WBC) enables us to not only classify the blood cells with deep learning but also quantitatively study their morphological and biochemical properties for hematology research. Introduction. Conventional methods for examining blood cells, such as blood smear analysis by medical professionals and fluorescence-activated cell sorting, require significant time, costs, and domain knowledge that could affect test results. While label-free imaging techniques that use a specimen's intrinsic contrast (e.g., multiphoton and Raman microscopy) have been used to characterize blood cells, their imaging procedures and instrumentations are relatively time-consuming and complex. Methods. The RI tomograms of the BM WBCs are acquired via Mach-Zehnder interferometer-based tomographic microscope and classified by a 3D convolutional neural network. We test our deep learning classifier for the four types of bone marrow WBC collected from healthy donors (n = 10 ): monocyte, myelocyte, B lymphocyte, and T lymphocyte. The quantitative parameters of WBC are directly obtained from the tomograms. Results. Our results show >99% accuracy for the binary classification of myeloids and lymphoids and >96% accuracy for the four-type classification of B and T lymphocytes, monocyte, and myelocytes. The feature learning capability of our approach is visualized via an unsupervised dimension reduction technique. Conclusion. We envision that the proposed cell classification framework can be easily integrated into existing blood cell investigation workflows, providing cost-effective and rapid diagnosis for hematologic malignancy.

Cell morphology-based machine learning models for human cell state classification

Herein, we implement and access machine learning architectures to ascertain models that differentiate healthy from apoptotic cells using exclusively forward (FSC) and side (SSC) scatter flow cytometry information. To generate training data, colorectal cancer HCT116 cells were subjected to miR-34a treatment and then classified using a conventional Annexin V/propidium iodide (PI)-staining assay. The apoptotic cells were defined as Annexin V-positive cells, which include early and late apoptotic cells, necrotic cells, as well as other dying or dead cells. In addition to fluorescent signal, we collected cell size and granularity information from the FSC and SSC parameters. Both parameters are subdivided into area, height, and width, thus providing a total of six numerical features that informed and trained our models. A collection of logistical regression, random forest, k-nearest neighbor, multilayer perceptron, and support vector machine was trained and tested for classification performance in predicting cell states using only the six aforementioned numerical features. Out of 1046 candidate models, a multilayer perceptron was chosen with 0.91 live precision, 0.93 live recall, 0.92 live f value and 0.97 live area under the ROC curve when applied on standardized data. We discuss and highlight differences in classifier performance and compare the results to the standard practice of forward and side scatter gating, typically performed to select cells based on size and/or complexity. We demonstrate that our model, a ready-to-use module for any flow cytometry-based analysis, can provide automated, reliable, and stain-free classification of healthy and apoptotic cells using exclusively size and granularity information.

Intelligent Platelet Morphometry

Technological advances in image-based platelet analysis or platelet morphometry are critical for a better understanding of the structure and function of platelets in biological research as well as for the development of better clinical strategies in medical practice. Recently, the advent of high-throughput optical imaging and deep learning has boosted platelet morphometry to the next level by providing a new set of capabilities beyond what is achievable with traditional platelet morphometry, shedding light on the unexplored domain of platelet analysis. This Opinion article introduces emerging opportunities in 'intelligent' platelet morphometry, which are expected to pave the way for a new class of diagnostics, pharmacometrics, and therapeutics.

AI on a chip

Artificial intelligence (AI) has dramatically changed the landscape of science, industry, defence, and medicine in the last several years. Supported by considerably enhanced computational power and cloud storage, the field of AI has shifted from mostly theoretical studies in the discipline of computer science to diverse real-life applications such as drug design, material discovery, speech recognition, self-driving cars, advertising, finance, medical imaging, and astronomical observation, where AI-produced outcomes have been proven to be comparable or even superior to the performance of human experts. In these applications, what is essentially important for the development of AI is the data needed for machine learning. Despite its prominent importance, the very first process of the AI development, namely data collection and data preparation, is typically the most laborious task and is often a limiting factor of constructing functional AI algorithms. Lab-on-a-chip technology, in particular microfluidics, is a powerful platform for both the construction and implementation of AI in a large-scale, cost-effective, high-throughput, automated, and multiplexed manner, thereby overcoming the above bottleneck. On this platform, high-throughput imaging is a critical tool as it can generate high-content information (e.g., size, shape, structure, composition, interaction) of objects on a large scale. High-throughput imaging can also be paired with sorting and DNA/RNA sequencing to conduct a massive survey of phenotype-genotype relations whose data is too complex to analyze with traditional computational tools, but is analyzable with the power of AI. In addition to its function as a data provider, lab-on-a-chip technology can also be employed to implement the developed AI for accurate identification, characterization, classification, and prediction of objects in mixed, heterogeneous, or unknown samples. In this review article, motivated by the excellent synergy between AI and lab-on-a-chip technology, we outline fundamental elements, recent advances, future challenges, and emerging opportunities of AI with lab-on-a-chip technology or "AI on a chip" for short.