Color calibration and fusion of lens-free and mobile-phone microscopy images for high-resolution and accurate color reproduction

Android mobile-platform-based image reconstruction for photoacoustic tomography

Significance

In photoacoustic tomography (PAT), numerous reconstruction algorithms have been utilized to recover initial pressure rise distribution from the acquired pressure waves. In practice, most of these reconstructions are carried out on a desktop/workstation and the mobile-based reconstructions are far-flung. In recent years, mobile phones are becoming so ubiquitous, and most of them encompass a higher computing ability. Hence, realizing PAT image reconstruction on a mobile platform is intrinsic, and it will enhance the adaptability of PAT systems with point-of-care applications.

Aim

To implement PAT image reconstruction in Android-based mobile platforms.

Approach

For implementing PAT image reconstruction in Android-based mobile platforms, we proposed an Android-based application using Python to perform beamforming process in Android phones.

Results

The performance of the developed application was analyzed on different mobile platforms using both simulated and experimental datasets. The results demonstrate that the developed algorithm can accomplish the image reconstruction of in vivo small animal brain dataset in 2.4 s. Furthermore, the developed application reconstructs PAT images with comparable speed and no loss of image quality compared to that on a laptop. Employing a two-fold downsampling procedure could serve as a viable solution for reducing the time needed for beamforming while preserving image quality with minimal degradation.

Conclusions

We proposed an Android-based application that achieves image reconstruction on cheap, small, and universally available phones instead of relatively bulky expensive desktop computers/laptops/workstations. A beamforming speed of 2.4 s is achieved without hampering the quality of the reconstructed image.

Portable, Automated and Deep-Learning-Enabled Microscopy for Smartphone-Tethered Optical Platform Towards Remote Homecare Diagnostics: A Review

Globally new pandemic diseases induce urgent demands for portable diagnostic systems to prevent and control infectious diseases. Smartphone-based portable diagnostic devices are significantly efficient tools to user-friendly connect personalized health conditions and collect valuable optical information for rapid diagnosis and biomedical research through at-home screening. Deep learning algorithms for portable microscopes also help to enhance diagnostic accuracy by reducing the imaging resolution gap between benchtop and portable microscopes. This review highlighted recent progress and continued efforts in a smartphone-tethered optical platform through portable, automated, and deep-learning-enabled microscopy for personalized diagnostics and remote monitoring. In detail, the optical platforms through smartphone-based microscopes and lens-free holographic microscopy are introduced, and deep learning-based portable microscopic imaging is explained to improve the image resolution and accuracy of diagnostics. The challenges and prospects of portable optical systems with microfluidic channels and a compact microscope to screen COVID-19 in the current pandemic are also discussed. It has been believed that this review offers a novel guide for rapid diagnosis, biomedical imaging, and digital healthcare with low cost and portability.

Nanozyme-based colorimetric biosensor with a systemic quantification algorithm for noninvasive glucose monitoring

Diabetes mellitus accompanies an abnormally high glucose level in the bloodstream. Early diagnosis and proper glycemic management of blood glucose are essential to prevent further progression and complications. Biosensor-based colorimetric detection has progressed and shown potential in portable and inexpensive daily assessment of glucose levels because of its simplicity, low-cost, and convenient operation without sophisticated instrumentation. Colorimetric glucose biosensors commonly use natural enzymes that recognize glucose and chromophores that detect enzymatic reaction products. However, many natural enzymes have inherent defects, limiting their extensive application. Recently, nanozyme-based colorimetric detection has drawn attention due to its merits including high sensitivity, stability under strict reaction conditions, flexible structural design with low-cost materials, and adjustable catalytic activities. This review discusses various nanozyme materials, colorimetric analytic methods and mechanisms, recent machine learning based analytic methods, quantification systems, applications and future directions for monitoring and managing diabetes.

van der Waals Semiconductor Empowered Vertical Color Sensor

Biomimetic artificial vision is receiving significant attention nowadays, particularly for the development of neuromorphic electronic devices, artificial intelligence, and microrobotics. Nevertheless, color recognition, the most critical vision function, is missed in the current research due to the difficulty of downscaling of the prevailing color sensing devices. Conventional color sensors typically adopt a lateral color sensing channel layout and consume a large amount of physical space, whereas compact designs suffer from an unsatisfactory color detection accuracy. In this work, we report a van der Waals semiconductor-empowered vertical color sensing structure with the emphasis on compact device profile and precise color recognition capability. More attractive, we endow color sensor hardware with the function of chromatic aberration correction, which can simplify the design of an optical lens system and, in turn, further downscales the artificial vision systems. Also, the dimension of a multiple pixel prototype device in our study confirms the scalability and practical potentials of our developed device architecture toward the above applications.

Thermotropic Liquid Crystals for Temperature Mapping

Wound management in Space is an important factor to be considered in future Human Space Exploration. It demands the development of reliable wound monitoring systems that will facilitate the assessment and proper care of wounds in isolated environments, such as Space. One possible system could be developed using liquid crystal films, which have been a promising solution for real-time in-situ temperature monitoring in healthcare, but they are not yet implemented in clinical practice. To progress in the latter, the goal of this study is twofold. First, it provides a full characterization of a sensing element composed of thermotropic liquid crystals arrays embedded between two elastomer layers, and second, it discusses how such a system compares against non-local infrared measurements. The sensing element evaluated here has an operating temperature range of 34–38°C, and a quick response time of approximately 0.25 s. The temperature distribution of surfaces obtained using this system was compared to the one obtained using the infrared thermography, a technique commonly used to measure temperature distributions at the wound site. This comparison was done on a mimicked wound, and results indicate that the proposed sensing element can reproduce the temperature distributions, similar to the ones obtained using infrared imaging. Although there is a long way to go before implementing the liquid crystal sensing element into clinical practice, the results of this work demonstrate that such sensors can be suitable for future wound monitoring systems.

Deep learning in optical metrology: a review

With the advances in scientific foundations and technological implementations, optical metrology has become versatile problem-solving backbones in manufacturing, fundamental research, and engineering applications, such as quality control, nondestructive testing, experimental mechanics, and biomedicine. In recent years, deep learning, a subfield of machine learning, is emerging as a powerful tool to address problems by learning from data, largely driven by the availability of massive datasets, enhanced computational power, fast data storage, and novel training algorithms for the deep neural network. It is currently promoting increased interests and gaining extensive attention for its utilization in the field of optical metrology. Unlike the traditional "physics-based" approach, deep-learning-enabled optical metrology is a kind of "data-driven" approach, which has already provided numerous alternative solutions to many challenging problems in this field with better performances. In this review, we present an overview of the current status and the latest progress of deep-learning technologies in the field of optical metrology. We first briefly introduce both traditional image-processing algorithms in optical metrology and the basic concepts of deep learning, followed by a comprehensive review of its applications in various optical metrology tasks, such as fringe denoising, phase retrieval, phase unwrapping, subset correlation, and error compensation. The open challenges faced by the current deep-learning approach in optical metrology are then discussed. Finally, the directions for future research are outlined.

Predicting global numbers of teleomorphic ascomycetes

Sexual reproduction is the basic way to form high genetic diversity and it is beneficial in evolution and speciation of fungi. The global diversity of teleomorphic species in Ascomycota has not been estimated. This paper estimates the species number for sexual ascomycetes based on five different estimation approaches, viz. by numbers of described fungi, by fungus:substrate ratio, by ecological distribution, by meta-DNA barcoding or culture-independent studies and by previous estimates of species in Ascomycota. The assumptions were made with the currently most accepted, “2.2–3.8 million” species estimate and results of previous studies concluding that 90% of the described ascomycetes reproduce sexually. The Catalogue of Life, Species Fungorum and published research were used for data procurement. The average value of teleomorphic species in Ascomycota from all methods is 1.86 million, ranging from 1.37 to 2.56 million. However, only around 83,000 teleomorphic species have been described in Ascomycota and deposited in data repositories. The ratio between described teleomorphic ascomycetes to predicted teleomorphic ascomycetes is 1:22. Therefore, where are the undiscovered teleomorphic ascomycetes? The undescribed species are no doubt to be found in biodiversity hot spots, poorly-studied areas and species complexes. Other poorly studied niches include extremophiles, lichenicolous fungi, human pathogens, marine fungi, and fungicolous fungi. Undescribed species are present in unexamined collections in specimen repositories or incompletely described earlier species. Nomenclatural issues, such as the use of separate names for teleomorph and anamorphs, synonyms, conspecific names, illegitimate and invalid names also affect the number of described species. Interspecies introgression results in new species, while species numbers are reduced by extinctions.

Evaluation of native canine skin color by smartphone-based dermatoscopy

BACKGROUND

Human skin color, predominantly determined by the chromophores of melanin, hemoglobin, and exogenous carotenoids, is often measured to serve various medical and cosmetic applications. Although colorimetry has been used to evaluate the skin erythema in allergic dogs, the native canine skin color remains unknown.

METHODS

We measured the skin color in 101 healthy dogs using a calibrated optical system with a smartphone and a mobile dermatoscope DermLite DL1. The results were retrieved in the CIELAB color system and compared to the human color ranges.

RESULTS

The lightness (L*) of canine skin ranged from 28.5 to 78.3, which is slightly broader than that of human skin. There was a difference of 3.9 in redness (a*) between canine and human skin, but this variation could be attributed to the similarly valued colorimetric error of the optical system. Nonetheless, the skin yellowness was significantly different for dogs and humans (respective median b* of 12.3 versus 16.6, p < 0.01). This difference might be due to canids not being able to accumulate typically yellowish carotenoids. Furthermore, the native canine skin color did not exhibit a typical dependence between the coordinates of lightness (L*) and yellowness (b*), known as the individual typology angle, °ITA.

CONCLUSION

We reported the first dataset of the native canine skin color in the CIELAB color space. We discovered a similarity in skin lightness and a difference in skin yellowness. However, further studies are needed for a more precise comparison of skin redness.

Cellphone enabled point-of-care assessment of breast tumor cytology and molecular HER2 expression from fine-needle aspirates

Management of breast cancer in limited-resource settings is hindered by a lack of low-cost, logistically sustainable approaches toward molecular and cellular diagnostic pathology services that are needed to guide therapy. To address these limitations, we have developed a multimodal cellphone-based platform—the EpiView-D4—that can evaluate both cellular morphology and molecular expression of clinically relevant biomarkers directly from fine-needle aspiration (FNA) of breast tissue specimens within 1 h. The EpiView-D4 is comprised of two components: (1) an immunodiagnostic chip built upon a “non-fouling” polymer brush-coating (the “D4”) which quantifies expression of protein biomarkers directly from crude cell lysates, and (2) a custom cellphone-based optical microscope (“EpiView”) designed for imaging cytology preparations and D4 assay readout. As a proof-of-concept, we used the EpiView-D4 for assessment of human epidermal growth factor receptor-2 (HER2) expression and validated the performance using cancer cell lines, animal models, and human tissue specimens. We found that FNA cytology specimens (prepared in less than 5 min with rapid staining kits) imaged by the EpiView-D4 were adequate for assessment of lesional cellularity and tumor content. We also found our device could reliably distinguish between HER2 expression levels across multiple different cell lines and animal xenografts. In a pilot study with human tissue (n = 19), we were able to accurately categorize HER2-negative and HER2-positve tumors from FNA specimens. Taken together, the EpiView-D4 offers a promising alternative to invasive—and often unavailable—pathology services and may enable the democratization of effective breast cancer management in limited-resource settings.

Smartphone-based colorimetric detection systems for glucose monitoring in the diagnosis and management of diabetes

Diabetes is a group of metabolic conditions resulting in high blood sugar levels over prolonged periods that affects hundreds of millions of patients worldwide. Measuring glucose concentration enables patient-specific insulin therapy, and is essential to reduce the severity of the disease, potential complications, and related mortalities. Recent advances and developments in smartphone-based colorimetric glucose detection systems are discussed in this review. The importance of glucose monitoring, data collection, transfer, and analysis, via non-invasive/invasive methods is highlighted. The review also presents various approaches using 3D-printed materials, screen-printed electrodes, polymer templates, designs allowing multiple glucose analysis, bioanalytes and/or nanostructures for glucose detection. The positive effects of advances in improving the performance of smartphone-based platforms are introduced along with future directions and trends in the application of emerging technologies in smartphone-based diagnostics.

Machine learning-enabled multiplexed microfluidic sensors

High-throughput, cost-effective, and portable devices can enhance the performance of point-of-care tests. Such devices are able to acquire images from samples at a high rate in combination with microfluidic chips in point-of-care applications. However, interpreting and analyzing the large amount of acquired data is not only a labor-intensive and time-consuming process, but also prone to the bias of the user and low accuracy. Integrating machine learning (ML) with the image acquisition capability of smartphones as well as increasing computing power could address the need for high-throughput, accurate, and automatized detection, data processing, and quantification of results. Here, ML-supported diagnostic technologies are presented. These technologies include quantification of colorimetric tests, classification of biological samples (cells and sperms), soft sensors, assay type detection, and recognition of the fluid properties. Challenges regarding the implementation of ML methods, including the required number of data points, image acquisition prerequisites, and execution of data-limited experiments are also discussed.

Accuracy of an Affordable Smartphone-Based Teledermoscopy System for Color Measurements in Canine Skin

Quality smartphone cameras and affordable dermatoscopes have enabled teledermoscopy to become a popular medical and veterinary tool for analyzing skin lesions such as melanoma and erythema. However, smartphones acquire images in an unknown RGB color space, which prevents a standardized colorimetric skin analysis. In this work, we supplemented a typical veterinary teledermoscopy system with a conventional color calibration procedure, and we studied two mid-priced smartphones in evaluating native and erythematous canine skin color. In a laboratory setting with the ColorChecker, the teledermoscopy system reached CIELAB-based color differences ΔE of 1.8-6.6 (CIE76) and 1.1-4.5 (CIE94). Intra- and inter-smartphone variability resulted in the color differences (CIE76) of 0.1, and 2.0-3.9, depending on the selected color range. Preliminary clinical measurements showed that canine skin is less red and yellow (lower a* and b* for ΔE of 10.7) than standard Caucasian human skin. Estimating the severity of skin erythema with an erythema index led to errors between 0.5-3%. After constructing a color calibration model for each smartphone, we expedited clinical measurements without losing colorimetric accuracy by introducing a simple image normalization on a white standard. To conclude, the calibrated teledermoscopy system is fast and accurate enough for various colorimetric applications in veterinary dermatology.

Disposable Sensors in Diagnostics, Food, and Environmental Monitoring

Disposable sensors are low-cost and easy-to-use sensing devices intended for short-term or rapid single-point measurements. The growing demand for fast, accessible, and reliable information in a vastly connected world makes disposable sensors increasingly important. The areas of application for such devices are numerous, ranging from pharmaceutical, agricultural, environmental, forensic, and food sciences to wearables and clinical diagnostics, especially in resource-limited settings. The capabilities of disposable sensors can extend beyond measuring traditional physical quantities (for example, temperature or pressure); they can provide critical chemical and biological information (chemo- and biosensors) that can be digitized and made available to users and centralized/decentralized facilities for data storage, remotely. These features could pave the way for new classes of low-cost systems for health, food, and environmental monitoring that can democratize sensing across the globe. Here, a brief insight into the materials and basics of sensors (methods of transduction, molecular recognition, and amplification) is provided followed by a comprehensive and critical overview of the disposable sensors currently used for medical diagnostics, food, and environmental analysis. Finally, views on how the field of disposable sensing devices will continue its evolution are discussed, including the future trends, challenges, and opportunities.

Optical refractometry using lensless holography and autofocusing

Conventional optical refractometry methods are often limited by a narrow measurement range, complex hardware, or relatively high cost. Here, we present a novel refractometry method to measure the bulk refractive index (RI) of materials (including solids and liquids) using lensless holographic on-chip imaging and autofocusing, which is simple, cost-effective, and has a large RI measurement range. As a proof of concept, two compact prototypes were built to measure the RIs of solid materials and liquids, respectively, and they were tested by measuring the RIs of a ZnSe plate and a microscopy immersion oil. Experimental results show that our devices have an average accuracy of ~3 × 10^-4 RI unit (RIU) with an estimated precision of ~3 × 10^-3 RIU for solids; and an average accuracy of ~1 × 10^-4 RIU with an estimated precision of ~3 × 10^-4 RIU for liquids. We believe that this cost-effective and portable RI measurement platform holds promise to be used in laboratory and industrial settings.

Current and Emerging Trends in Point-of-Care Technology and Strategies for Clinical Validation and Implementation

BACKGROUND

Point-of-care technology (POCT) provides actionable information at the site of care to allow rapid clinical decision-making. With healthcare emphasis shifting toward precision medicine, population health, and chronic disease management, the potential impact of POCT continues to grow, and several prominent POCT trends have emerged or strengthened in the last decade.

CONTENT

This review summarizes current and emerging trends in POCT, including technologies approved or cleared by the Food and Drug Administration or in development. Technologies included have either impacted existing clinical diagnostics applications (e.g., continuous monitoring and targeted nucleic acid testing) or are likely to impact diagnostics delivery in the near future. The focus is limited to in vitro diagnostics applications, although in some sections, technologies beyond in vitro diagnostics are also included given the commonalities (e.g., ultrasound plug-ins for smart phones). For technologies in development (e.g., wearables, noninvasive testing, mass spectrometry and nuclear magnetic resonance, paper-based diagnostics, nanopore-based devices, and digital microfluidics), we also discuss their potential clinical applications and provide perspectives on strategies beyond technological and analytical proof of concept, with the end goal of clinical implementation and impact.

SUMMARY

The field of POCT has witnessed strong growth over the past decade, as evidenced by new clinical or consumer products or research and development directions. Combined with the appropriate strategies for clinical needs assessment, validation, and implementation, these and future POCTs may significantly impact care delivery and associated outcomes and costs.

Lensless digital holographic microscopy and its applications in biomedicine and environmental monitoring

Optical compound microscope has been a major tool in biomedical imaging for centuries. Its performance relies on relatively complicated, bulky and expensive lenses and alignment mechanics. In contrast, the lensless microscope digitally reconstructs microscopic images of specimens without using any lenses, as a result of which it can be made much smaller, lighter and lower-cost. Furthermore, the limited space-bandwidth product of objective lenses in a conventional microscope can be significantly surpassed by a lensless microscope. Such lensless imaging designs have enabled high-resolution and high-throughput imaging of specimens using compact, portable and cost-effective devices to potentially address various point-of-care, global-health and telemedicine related challenges. In this review, we discuss the operation principles and the methods behind lensless digital holographic on-chip microscopy. We also go over various applications that are enabled by cost-effective and compact implementations of lensless microscopy, including some recent work on air quality monitoring, which utilized machine learning for high-throughput and accurate quantification of particulate matter in air. Finally, we conclude with a brief future outlook of this computational imaging technology.

3D imaging of optically cleared tissue using a simplified CLARITY method and on-chip microscopy

High-throughput sectioning and optical imaging of tissue samples using traditional immunohistochemical techniques can be costly and inaccessible in resource-limited areas. We demonstrate three-dimensional (3D) imaging and phenotyping in optically transparent tissue using lens-free holographic on-chip microscopy as a low-cost, simple, and high-throughput alternative to conventional approaches. The tissue sample is passively cleared using a simplified CLARITY method and stained using 3,3'-diaminobenzidine to target cells of interest, enabling bright-field optical imaging and 3D sectioning of thick samples. The lens-free computational microscope uses pixel super-resolution and multi-height phase recovery algorithms to digitally refocus throughout the cleared tissue and obtain a 3D stack of complex-valued images of the sample, containing both phase and amplitude information. We optimized the tissue-clearing and imaging system by finding the optimal illumination wavelength, tissue thickness, sample preparation parameters, and the number of heights of the lens-free image acquisition and implemented a sparsity-based denoising algorithm to maximize the imaging volume and minimize the amount of the acquired data while also preserving the contrast-to-noise ratio of the reconstructed images. As a proof of concept, we achieved 3D imaging of neurons in a 200-μm-thick cleared mouse brain tissue over a wide field of view of 20.5 mm2. The lens-free microscope also achieved more than an order-of-magnitude reduction in raw data compared to a conventional scanning optical microscope imaging the same sample volume. Being low cost, simple, high-throughput, and data-efficient, we believe that this CLARITY-enabled computational tissue imaging technique could find numerous applications in biomedical diagnosis and research in low-resource settings.

Rapid in-focus corrections on quantitative amplitude and phase imaging using transport of intensity equation method

Transport of intensity equation (TIE) method can acquire sample phase distributions with high speed and accuracy, offering another perspective for cellular observations and measurements. However, caused by incorrect focal plane determination, blurs and halos are induced, decreasing resolution and accuracy in both retrieved amplitude and phase information. In order to obtain high-accurate sample details, we propose TIE based in-focus correction technique for quantitative amplitude and phase imaging, which can locate focal plane and then retrieve both in-focus intensity and phase distributions combining with numerical wavefront extraction and propagation as well as physical image recorder translation. Certified by both numerical simulations and practical measurements, it is believed the proposed method not only captures high-accurate in-focus sample information, but also provides a potential way for fast autofocusing in microscopic system.

Smartphone based hand-held quantitative phase microscope using the transport of intensity equation method

In order to realize high contrast imaging with portable devices for potential mobile healthcare, we demonstrate a hand-held smartphone based quantitative phase microscope using the transport of intensity equation method. With a cost-effective illumination source and compact microscope system, multi-focal images of samples can be captured by the smartphone's camera via manual focusing. Phase retrieval is performed using a self-developed Android application, which calculates sample phases from multi-plane intensities via solving the Poisson equation. We test the portable microscope using a random phase plate with known phases, and to further demonstrate its performance, a red blood cell smear, a Pap smear and monocot root and broad bean epidermis sections are also successfully imaged. Considering its advantages as an accurate, high-contrast, cost-effective and field-portable device, the smartphone based hand-held quantitative phase microscope is a promising tool which can be adopted in the future in remote healthcare and medical diagnosis.