Increasing a microscope's effective field of view via overlapped imaging and machine learning

Neural network-based processing and reconstruction of compromised biophotonic image data

In recent years, the integration of deep learning techniques with biophotonic setups has opened new horizons in bioimaging. A compelling trend in this field involves deliberately compromising certain measurement metrics to engineer better bioimaging tools in terms of e.g., cost, speed, and form-factor, followed by compensating for the resulting defects through the utilization of deep learning models trained on a large amount of ideal, superior or alternative data. This strategic approach has found increasing popularity due to its potential to enhance various aspects of biophotonic imaging. One of the primary motivations for employing this strategy is the pursuit of higher temporal resolution or increased imaging speed, critical for capturing fine dynamic biological processes. Additionally, this approach offers the prospect of simplifying hardware requirements and complexities, thereby making advanced imaging standards more accessible in terms of cost and/or size. This article provides an in-depth review of the diverse measurement aspects that researchers intentionally impair in their biophotonic setups, including the point spread function (PSF), signal-to-noise ratio (SNR), sampling density, and pixel resolution. By deliberately compromising these metrics, researchers aim to not only recuperate them through the application of deep learning networks, but also bolster in return other crucial parameters, such as the field of view (FOV), depth of field (DOF), and space-bandwidth product (SBP). Throughout this article, we discuss various biophotonic methods that have successfully employed this strategic approach. These techniques span a wide range of applications and showcase the versatility and effectiveness of deep learning in the context of compromised biophotonic data. Finally, by offering our perspectives on the exciting future possibilities of this rapidly evolving concept, we hope to motivate our readers from various disciplines to explore novel ways of balancing hardware compromises with compensation via artificial intelligence (AI).

A miniaturized mesoscope for the large-scale single-neuron-resolved imaging of neuronal activity in freely behaving mice

Exploring the relationship between neuronal dynamics and ethologically relevant behaviour involves recording neuronal-population activity using technologies that are compatible with unrestricted animal behaviour. However, head-mounted microscopes that accommodate weight limits to allow for free animal behaviour typically compromise field of view, resolution or depth range, and are susceptible to movement-induced artefacts. Here we report a miniaturized head-mounted fluorescent mesoscope that we systematically optimized for calcium imaging at single-neuron resolution, for increased fields of view and depth of field, and for robustness against motion-generated artefacts. Weighing less than 2.5 g, the mesoscope enabled recordings of neuronal-population activity at up to 16 Hz, with 4 μm resolution over 300 μm depth-of-field across a field of view of 3.6 × 3.6 mm2 in the cortex of freely moving mice. We used the mesoscope to record large-scale neuronal-population activity in socially interacting mice during free exploration and during fear-conditioning experiments, and to investigate neurovascular coupling across multiple cortical regions.

The Impact of Artificial Intelligence on Microbial Diagnosis

Traditional microbial diagnostic methods face many obstacles such as sample handling, culture difficulties, misidentification, and delays in determining susceptibility. The advent of artificial intelligence (AI) has markedly transformed microbial diagnostics with rapid and precise analyses. Nonetheless, ethical considerations accompany AI adoption, necessitating measures to uphold patient privacy, mitigate biases, and ensure data integrity. This review examines conventional diagnostic hurdles, stressing the significance of standardized procedures in sample processing. It underscores AI's significant impact, particularly through machine learning (ML), in microbial diagnostics. Recent progressions in AI, particularly ML methodologies, are explored, showcasing their influence on microbial categorization, comprehension of microorganism interactions, and augmentation of microscopy capabilities. This review furnishes a comprehensive evaluation of AI's utility in microbial diagnostics, addressing both advantages and challenges. A few case studies including SARS-CoV-2, malaria, and mycobacteria serve to illustrate AI's potential for swift and precise diagnosis. Utilization of convolutional neural networks (CNNs) in digital pathology, automated bacterial classification, and colony counting further underscores AI's versatility. Additionally, AI improves antimicrobial susceptibility assessment and contributes to disease surveillance, outbreak forecasting, and real-time monitoring. Despite a few limitations, integration of AI in diagnostic microbiology presents robust solutions, user-friendly algorithms, and comprehensive training, promising paradigm-shifting advancements in healthcare.

Large depth-of-field ultra-compact microscope by progressive optimization and deep learning

The optical microscope is customarily an instrument of substantial size and expense but limited performance. Here we report an integrated microscope that achieves optical performance beyond a commercial microscope with a 5×, NA 0.1 objective but only at 0.15 cm³ and 0.5 g, whose size is five orders of magnitude smaller than that of a conventional microscope. To achieve this, a progressive optimization pipeline is proposed which systematically optimizes both aspherical lenses and diffractive optical elements with over 30 times memory reduction compared to the end-to-end optimization. By designing a simulation-supervision deep neural network for spatially varying deconvolution during optical design, we accomplish over 10 times improvement in the depth-of-field compared to traditional microscopes with great generalization in a wide variety of samples. To show the unique advantages, the integrated microscope is equipped in a cell phone without any accessories for the application of portable diagnostics. We believe our method provides a new framework for the design of miniaturized high-performance imaging systems by integrating aspherical optics, computational optics, and deep learning.

Single-shot extended field of view imaging using point spread function engineering

We present a single-shot computational imaging system employing pupil phase engineering to extend the field of view (FOV) beyond the physical sensor limit. Our approach uses a point spread function in the form of a multiple-point impulse response (MPIR). Unlike the traditional point-to-point imaging model used by most traditional optical imaging systems, the proposed MPIR model can collect information from within and outside the sensor boundary. The detected raw image despite being scrambled can be decoded via a sparse optimization algorithm to get extended FOV imaging performance. We provide a thorough analysis of MPIR design regarding the number of impulses and their spatial extent. Increasing the number of impulses in MPIR of a given spatial extent leads to better information gathering within the detector region; however, it also reduces contrast in the raw data. Therefore, a trade-off between increasing the information and keeping adequate contrast in the detected data is necessary to achieve high-quality reconstruction. We first illustrate this trade-off with a simulation study and present experimental results on a suitably designed extended FOV imaging system. We demonstrate reconstructed images with a 4× gain in pixels over the native detection area without loss of spatial resolution. The proposed system design considerations are generic and can be applied to various imaging systems for extended FOV performance.

Artificial intelligence-based tools applied to pathological diagnosis of microbiological diseases

Infectious diseases still threaten the global community, especially in resource-limited countries. An accurate diagnosis is paramount to proper patient and public health management. Identification of many microbes still relies on manual microscopic examination, a time-consuming process requiring skilled staff. Thus, artificial intelligence (AI) has been exploited for identification of microorganisms. A systematic search was carried out using electronic databases looking for studies dealing with the application of AI to pathology microbiology specimens. Of 4596 retrieved articles, 110 were included. The main applications of AI regarded malaria (54 studies), bacteria (28), nematodes (14), and other protozoa (11). Most publications examined cytological material (95, 86%), mainly analyzing images acquired through microscope cameras (65, 59%) or coupled with smartphones (16, 15%). Various deep-learning strategies were used for the analysis of digital images, achieving highly satisfactory results. The published evidence suggests that AI can be reliably utilized for assisting pathologists in the detection of microorganisms. Further technologic improvement and availability of datasets for training AI-based algorithms would help expand this field and widen its adoption, especially for developing countries.

Automatic whole blood cell analysis from blood smear using label-free multi-modal imaging with deep neural networks

Whole blood cell analysis is widely used in medical applications since its results are indicators for diagnosing a series of diseases. In this work, we report automatic whole blood cell analysis from blood smear using label-free multi-modal imaging with deep neural networks. First, a commercial microscope equipped with our developed Phase Real-time Microscope Camera (PhaseRMiC) obtains both bright-field and quantitative phase images. Then, these images are automatically processed by our designed blood smear recognition networks (BSRNet) that recognize erythrocytes, leukocytes and platelets. Finally, blood cell parameters such as counts, shapes and volumes can be extracted according to both quantitative phase images and automatic recognition results. The proposed whole blood cell analysis technique provides high-quality blood cell images and supports accurate blood cell recognition and analysis. Moreover, this approach requires rather simple and cost-effective setups as well as easy and rapid sample preparations. Therefore, this proposed method has great potential application in blood testing aiming at disease diagnostics.

Computational optical sensing and imaging 2021: feature issue introduction

This Feature Issue includes 2 reviews and 34 research articles that highlight recent works in the field of Computational Optical Sensing and Imaging. Many of the works were presented at the 2021 OSA Topical Meeting on Computational Optical Sensing and Imaging, held virtually from July 19 to July 23, 2021. Articles in the feature issue cover a broad scope of computational imaging topics, such as microscopy, 3D imaging, phase retrieval, non-line-of-sight imaging, imaging through scattering media, ghost imaging, compressed sensing, and applications with new types of sensors. Deep learning approaches for computational imaging and sensing are also a focus of this feature issue.

Computational Optical Sensing and Imaging 2021: introduction to the feature issue

This feature issue includes two reviews and 34 research papers that highlight recent works in the field of computational optical sensing and imaging. Many of the works were presented at the 2021 Optica (formerly OSA) Topical Meeting on Computational Optical Sensing and Imaging, held virtually from 19 July to 23 July 2021. Papers in the feature issue cover a broad scope of computational imaging topics, such as microscopy, 3D imaging, phase retrieval, non-line-of-sight imaging, imaging through scattering media, ghost imaging, compressed sensing, and applications with new types of sensors. Deep learning approaches for computational imaging and sensing are also a focus of this feature issue.