Deep learning a boon for biophotonics?

On-site SERS analysis and intelligent multi-identification of fentanyl class substances by deep machine learning

As the types of fentanyl class substances continue to grow, a universal SERS sensor is essential for the application of discriminant detection of fentanyl substances. A new nanomaterial SERS sensor-Ag@Au NPs-paper was developed. The SERS sensitivity and stability of Ag@Au NPs-paper were investigated by using R6G molecule, and the results showed that Ag@Au NPs-paper has excellent performance. In combination with visual analysis and machine learning methods, Ag@Au NPs-paper has been successfully applied to the analysis of fentanyl class substances and the component identification of binary fentanyl mixtures, and thus it can be effectively used in food safety, environmental toxicants and other fields.

Enhancing decision confidence in AI using Monte Carlo dropout for Raman spectra classification

BACKGROUND

Machine learning algorithms for bacterial strain identification using Raman spectroscopy have been widely used in microbiology. During the training phase, existing datasets are augmented and used to optimize model architecture and hyperparameters. After training, it is presumed that the models have reached their peak performance and are used for inference without being further enhanced. Our methodology combines Monte Carlo Dropout (MCD) with convolutional neural networks (CNNs) by utilizing dropout during the inference phase, which enables to measure the model uncertainty, a critical but often ignored aspect in deep learning models.

RESULTS

We categorize unseen input data into two subsets based on the uncertainty of their prediction by employing MCD and defining the threshold using the Gaussian Mixture Model (GMM). The final prediction is obtained on the subset of testing data that exhibits lower model uncertainty, thereby enhancing the reliability of the results. To validate our method, we applied it to two Raman spectra datasets. As a result, we have observed an increase in accuracy of 9 % for Dataset 1 (from 83.10 % to 92.10 %) and 12.82 % for Dataset 2 (from 83.86 % to 96.68 %). These improvements were observed within specific subsets of the data: 826 out of 1206 spectra in Dataset 1 and 1700 out of 3000 spectra in Dataset 2. This demonstrates the effectiveness of our approach in improving prediction accuracy by focusing on data with lower uncertainty.

SIGNIFICANCE

Different from routine prediction based on mere probabilities, we believe this uncertainty-guided prediction is more effective to ensure a high prediction rate rather than the prediction on the entire dataset. By guiding the decision-making of a model on higher-confidence subsets, our methodology can enhance the accuracy of classification in critical areas like disease diagnosis and safety monitoring. This targeted approach is to advance microbial identification and produces more trustworthy predictions.

Endomicroscopic AI-driven morphochemical imaging and fs-laser ablation for selective tumor identification and selective tissue removal

The rising incidence of head and neck cancer represents a serious global health challenge, requiring more accurate diagnosis and innovative surgical approaches. Multimodal nonlinear optical microscopy, combining coherent anti-Stokes Raman scattering (CARS), two-photon excited fluorescence (TPEF), and second-harmonic generation (SHG) with deep learning-based analysis routines, offers label-free assessment of the tissue's morphochemical composition and allows early-stage and automatic detection of disease. For clinical intraoperative application, compact devices are required. In this preclinical study, a cohort of 15 patients was examined with a newly developed rigid CARS/TPEF/SHG endomicroscope. To detect head and neck tumor from the multimodal data, deep learning-based semantic segmentation models were used. This preclinical study yields in a diagnostic sensitivity of 88% and a specificity of 96%. To combine diagnostics with therapy, machine learning-inspired image-guided selective tissue removal was used by integrating femtosecond laser ablation into the endomicroscope. This enables a powerful approach of intraoperative "seek and treat," paving the way to advanced surgical treatment.

Spectral Zones-Based SHAP/LIME: Enhancing Interpretability in Spectral Deep Learning Models Through Grouped Feature Analysis

Interpretability is just as important as accuracy when it comes to complex models, especially in the context of deep learning models. Explainable artificial intelligence (XAI) approaches have been developed to address this problem. The literature on XAI for spectroscopy mainly emphasizes independent feature analysis with limited application of zone analysis. Individual feature analysis methods, such as shapley additive explanations (SHAP) and local interpretable model-agnostic explanations (LIME), have limitations due to their dependence on perturbations. These methods measure how AI models respond to sudden changes in the individual feature values. While they can help identify the most impactful features, the abrupt shifts introduced by replacing these values with zero or the expected ones may not accurately represent real-world scenarios. This can lead to mathematical and computational interpretations that are neither physically realistic nor intuitive to humans. Our proposed method does not rely on individual disturbances. Instead, it targets "spectral zones" to directly estimate the effect of group disturbances on a trained model. Consequently, factors such as sample size, hyperparameter selection, and other training-related considerations are not the primary focus of the XAI methods. To achieve this, we have developed a modified version of LIME and SHAP capable of performing group perturbations, enhancing explainability and realism while minimizing noise in the plots used for interpretability. Additionally, we employed an efficient approach to calculate spectral zones for complex spectra with indistinct spectral boundaries. Users can also define the zones themselves using their domain-specific knowledge.

Neural network-based processing and reconstruction of compromised biophotonic image data

In recent years, the integration of deep learning techniques with biophotonic setups has opened new horizons in bioimaging. A compelling trend in this field involves deliberately compromising certain measurement metrics to engineer better bioimaging tools in terms of e.g., cost, speed, and form-factor, followed by compensating for the resulting defects through the utilization of deep learning models trained on a large amount of ideal, superior or alternative data. This strategic approach has found increasing popularity due to its potential to enhance various aspects of biophotonic imaging. One of the primary motivations for employing this strategy is the pursuit of higher temporal resolution or increased imaging speed, critical for capturing fine dynamic biological processes. Additionally, this approach offers the prospect of simplifying hardware requirements and complexities, thereby making advanced imaging standards more accessible in terms of cost and/or size. This article provides an in-depth review of the diverse measurement aspects that researchers intentionally impair in their biophotonic setups, including the point spread function (PSF), signal-to-noise ratio (SNR), sampling density, and pixel resolution. By deliberately compromising these metrics, researchers aim to not only recuperate them through the application of deep learning networks, but also bolster in return other crucial parameters, such as the field of view (FOV), depth of field (DOF), and space-bandwidth product (SBP). Throughout this article, we discuss various biophotonic methods that have successfully employed this strategic approach. These techniques span a wide range of applications and showcase the versatility and effectiveness of deep learning in the context of compromised biophotonic data. Finally, by offering our perspectives on the exciting future possibilities of this rapidly evolving concept, we hope to motivate our readers from various disciplines to explore novel ways of balancing hardware compromises with compensation via artificial intelligence (AI).

Accelerating whole-sample polarization-resolved second harmonic generation imaging in mammary gland tissue via generative adversarial networks

Polarization second harmonic generation (P-SHG) imaging is a powerful technique for studying the structure and properties of biological and material samples. However, conventional whole-sample P-SHG imaging is time consuming and requires expensive equipment. This paper introduces a novel approach that significantly improves imaging resolution under conditions of reduced imaging time and resolution, utilizing enhanced super-resolution generative adversarial networks (ESRGAN) to upscale low-resolution images. We demonstrate that this innovative approach maintains high image quality and analytical accuracy, while reducing the imaging time by more than 95%. We also discuss the benefits of the proposed method for reducing laser-induced photodamage, lowering the cost of optical components, and increasing the accessibility and applicability of P-SHG imaging in various fields. Our work significantly advances whole-sample mammary gland P-SHG imaging and opens new possibilities for scientific discovery and innovation.

Improving skin cancer detection by Raman spectroscopy using convolutional neural networks and data augmentation

Background

Our previous studies have demonstrated that Raman spectroscopy could be used for skin cancer detection with good sensitivity and specificity. The objective of this study is to determine if skin cancer detection can be further improved by combining deep neural networks and Raman spectroscopy.

Patients and methods

Raman spectra of 731 skin lesions were included in this study, containing 340 cancerous and precancerous lesions (melanoma, basal cell carcinoma, squamous cell carcinoma and actinic keratosis) and 391 benign lesions (melanocytic nevus and seborrheic keratosis). One-dimensional convolutional neural networks (1D-CNN) were developed for Raman spectral classification. The stratified samples were divided randomly into training (70%), validation (10%) and test set (20%), and were repeated 56 times using parallel computing. Different data augmentation strategies were implemented for the training dataset, including added random noise, spectral shift, spectral combination and artificially synthesized Raman spectra using one-dimensional generative adversarial networks (1D-GAN). The area under the receiver operating characteristic curve (ROC AUC) was used as a measure of the diagnostic performance. Conventional machine learning approaches, including partial least squares for discriminant analysis (PLS-DA), principal component and linear discriminant analysis (PC-LDA), support vector machine (SVM), and logistic regression (LR) were evaluated for comparison with the same data splitting scheme as the 1D-CNN.

Results

The ROC AUC of the test dataset based on the original training spectra were 0.886±0.022 (1D-CNN), 0.870±0.028 (PLS-DA), 0.875±0.033 (PC-LDA), 0.864±0.027 (SVM), and 0.525±0.045 (LR), which were improved to 0.909±0.021 (1D-CNN), 0.899±0.022 (PLS-DA), 0.895±0.022 (PC-LDA), 0.901±0.020 (SVM), and 0.897±0.021 (LR) respectively after augmentation of the training dataset (p<0.0001, Wilcoxon test). Paired analyses of 1D-CNN with conventional machine learning approaches showed that 1D-CNN had a 1-3% improvement (p<0.001, Wilcoxon test).

Conclusions

Data augmentation not only improved the performance of both deep neural networks and conventional machine learning techniques by 2-4%, but also improved the performance of the models on spectra with higher noise or spectral shifting. Convolutional neural networks slightly outperformed conventional machine learning approaches for skin cancer detection by Raman spectroscopy.

INSTRAS: INfrared Spectroscopic imaging-based TRAnsformers for medical image Segmentation

Infrared (IR) spectroscopic imaging is of potentially wide use in medical imaging applications due to its ability to capture both chemical and spatial information. This complexity of the data both necessitates using machine intelligence as well as presents an opportunity to harness a high-dimensionality data set that offers far more information than today's manually-interpreted images. While convolutional neural networks (CNNs), including the well-known U-Net model, have demonstrated impressive performance in image segmentation, the inherent locality of convolution limits the effectiveness of these models for encoding IR data, resulting in suboptimal performance. In this work, we propose an INfrared Spectroscopic imaging-based TRAnsformers for medical image Segmentation (INSTRAS). This novel model leverages the strength of the transformer encoders to segment IR breast images effectively. Incorporating skip-connection and transformer encoders, INSTRAS overcomes the issue of pure convolution models, such as the difficulty of capturing long-range dependencies. To evaluate the performance of our model and existing convolutional models, we conducted training on various encoder-decoder models using a breast dataset of IR images. INSTRAS, utilizing 9 spectral bands for segmentation, achieved a remarkable AUC score of 0.9788, underscoring its superior capabilities compared to purely convolutional models. These experimental results attest to INSTRAS's advanced and improved segmentation abilities for IR imaging.

Combined Mutual Learning Net for Raman Spectral Microbial Strain Identification

Infectious diseases pose a significant threat to global health, yet traditional microbiological identification methods suffer from drawbacks, such as high costs and long processing times. Raman spectroscopy, a label-free and noninvasive technique, provides rich chemical information and has tremendous potential in fast microbial diagnoses. Here, we propose a novel Combined Mutual Learning Net that precisely identifies microbial subspecies. It demonstrated an average identification accuracy of 87.96% in an open-access data set with thirty microbial strains, representing a 5.76% improvement. 50% of the microbial subspecies accuracies were elevated by 1% to 46%, especially for E. coli 2 improved from 31% to 77%. Furthermore, it achieved a remarkable subspecies accuracy of 92.4% in the custom-built fiber-optical tweezers Raman spectroscopy system, which collects Raman spectra at a single-cell level. This advancement demonstrates the effectiveness of this method in microbial subspecies identification, offering a promising solution for microbiology diagnosis.

Efficacy of tyrosine kinase inhibitors examined by a combination of Raman micro-spectroscopy and a deep wavelet scattering-based multivariate analysis framework

HER2 is a crucial therapeutic target in breast cancer, and the survival rate of breast cancer patients has increased because of this receptor's inhibition. However, tumors have shown resistance to this therapeutic strategy due to oncogenic mutations that decrease the binding of several HER2-targeted drugs, including lapatinib, and confer resistance to this drug. Neratinib can overcome this drug resistance and effectively inhibit HER2 signaling and tumor growth. In the present study, we examined the efficacy of lapatinib and neratinib using breast cancer cells by Raman microscopy combined with a deep wavelet scattering-based multivariate analysis framework. This approach discriminated between control cells and drug-treated cells with high accuracy, compared to classical principal component analysis. Both lapatinib and neratinib induced changes in the cellular biochemical composition. Furthermore, the Raman results were compared with the results of several in vitro assays. For instance, drug-treated cells exhibited (i) inhibition of ERK and AKT phosphorylation, (ii) inhibition of cellular proliferation, (iii) cell-cycle arrest, and (iv) apoptosis as indicated by western blotting, real-time cell analysis (RTCA), cell-cycle analysis, and apoptosis assays. Thus, the observed Raman spectral changes are attributed to cell-cycle arrest and apoptosis. The results also indicated that neratinib is more potent than lapatinib. Moreover, the uptake and distribution of lapatinib in cells were visualized through its label-free marker bands in the fingerprint region using Raman spectral imaging. These results show the prospects of Raman microscopy in drug evaluation and presumably in drug discovery.

Siamese Networks for Clinically Relevant Bacteria Classification Based on Raman Spectroscopy

Identifying bacterial strains is essential in microbiology for various practical applications, such as disease diagnosis and quality monitoring of food and water. Classical machine learning algorithms have been utilized to identify bacteria based on their Raman spectra. However, convolutional neural networks (CNNs) offer higher classification accuracy, but they require extensive training sets and retraining of previous untrained class targets can be costly and time-consuming. Siamese networks have emerged as a promising solution. They are composed of two CNNs with the same structure and a final network that acts as a distance metric, converting the classification problem into a similarity problem. Classical machine learning approaches, shallow and deep CNNs, and two Siamese network variants were tailored and tested on Raman spectral datasets of bacteria. The methods were evaluated based on mean sensitivity, training time, prediction time, and the number of parameters. In this comparison, Siamese-model2 achieved the highest mean sensitivity of 83.61 ± 4.73 and demonstrated remarkable performance in handling unbalanced and limited data scenarios, achieving a prediction accuracy of 73%. Therefore, the choice of model depends on the specific trade-off between accuracy, (prediction/training) time, and resources for the particular application. Classical machine learning models and shallow CNN models may be more suitable if time and computational resources are a concern. Siamese networks are a good choice for small datasets and CNN for extensive data.

Label-free SERS detection of prostate cancer based on multi-layer perceptron surrogate model method

Prior surface-enhanced Raman spectroscopy (SERS) research has shown that pre-processing is necessary before analysis. Pre-processing also typically serves the dual purposes of removing the auto-fluorescence background and minimizing data volatility. This method allows for a more accurate comparison of spectral traits and relative SERS peak strength. However, because there are so many different kinds of samples, it can take a long time, and there is no assurance that the approach chosen will work well with a particular kind of sample. Therefore, this study employed a deep learning technique called multi-layer perceptron (MLP) to simplify the pre-processing of blood plasma SERS samples in patients with prostate cancer (PC), as well as to enhance the sensitivity and specificity of diagnosis using SERS technology. First of all, significant variations in peak intensity can be observed in the difference spectra, facilitating differentiation between PC and normal groups. Second, the data analysis was carried out in three different stages (raw data, defluorescenced data, and normalized data) using principal component analysis and linear discriminant analysis (PCA-LDA), as well as PCA-multi-layer perceptron (PCA-MLP). Finally, when SERS data was analyzed using PCA-LDA, there were significant differences in classification accuracy across each stage (The classification accuracy of three different stages were 76.90%, 85.60%, 95.20%, respectively). However, when PCA-MLP was utilized for SERS data analysis, the classification accuracy remained consistently high and stable (The classification accuracy of three different stages were 92.00%, 92.40%, 96.70%, respectively). The experimental results of PCA-MLP for classifying specific SERS data indicate that analyzing raw data directly can simplify the experimental process and enhance the efficacy of SERS analysis.

Biophotonics as a new application in optical technology: A bibliometric analysis

Biophotonics procures wide practicability in life sciences and medicines. The contribution of biophotonics is well recognized in various Nobel Prizes. Therefore, this paper aims to conduct a bibliometric analysis of biophotonics publications. The scientific database used is the Web of Science database. Harzing's Publish or Perish and VOSviewer are the bibliometric tools used in this analysis. This study found an increasing trend in the number of publications in recent years as the number of publications peaked at 347 publications in 2020. Most of the documents are articles (3361 publications) and proceeding papers (1632 publications). The top three subject areas are Optics (3206 publications), Engineering (1706 publications) and Radiology, Nuclear Medicine, and Medical Imaging (1346 publications). The United States has the highest number of publications (2041 publications) and citation impact (38.07 citations per publication; h-index: 125). The top three publication titles are Proceedings of SPIE (920 publications), Journal of Biomedical Optics (599 publications), and Proceedings of the Society of Photo Optical Instrumentation Engineers SPIE (245 publications). The potential areas for future research include to overcome the optical penetration depth issue and to develop publicly available biosensors for the detection of common diseases.

A review of recent progress in the application of Raman spectroscopy and SERS detection of microplastics and derivatives

The global environmental concern surrounding microplastic (MP) pollution has raised alarms due to its potential health risks to animals, plants, and humans. Because of the complex structure and composition of microplastics (MPs), the detection methods are limited, resulting in restricted detection accuracy. Surface enhancement of Raman spectroscopy (SERS), a spectral technique, offers several advantages, such as high resolution and low detection limit. It has the potential to be extensively employed for sensitive detection and high-resolution imaging of microplastics. We have summarized the research conducted in recent years on the detection of microplastics using Raman and SERS. Here, we have reviewed qualitative and quantitative analyses of microplastics and their derivatives, as well as the latest progress, challenges, and potential applications.

Dimensionality reduction for deep learning in infrared microscopy: a comparative computational survey

While infrared microscopy provides molecular information at spatial resolution in a label-free manner, exploiting both spatial and molecular information for classifying the disease status of tissue samples constitutes a major challenge. One strategy to mitigate this problem is to embed high-dimensional pixel spectra in lower dimensions, aiming to preserve molecular information in a more compact manner, which reduces the amount of data and promises to make subsequent disease classification more accessible for machine learning procedures. In this study, we compare several dimensionality reduction approaches and their effect on identifying cancer in the context of a colon carcinoma study. We observe surprisingly small differences between convolutional neural networks trained on dimensionality reduced spectra compared to utilizing full spectra, indicating a clear tendency of the convolutional networks to focus on spatial rather than spectral information for classifying disease status.

Computational approaches in rheumatic diseases - Deciphering complex spatio-temporal cell interactions

Inflammatory arthritis, including rheumatoid (RA), and psoriatic (PsA) arthritis, are clinically and immunologically heterogeneous diseases with no identified cure. Chronic inflammation of the synovial tissue ushers loss of function of the joint that severely impacts the patient's quality of life, eventually leading to disability and life-threatening comorbidities. The pathogenesis of synovial inflammation is the consequence of compounded immune and stromal cell interactions influenced by genetic and environmental factors. Deciphering the complexity of the synovial cellular landscape has accelerated primarily due to the utilisation of bulk and single cell RNA sequencing. Particularly the capacity to generate cell-cell interaction networks could reveal evidence of previously unappreciated processes leading to disease. However, there is currently a lack of universal nomenclature as a result of varied experimental and technological approaches that discombobulates the study of synovial inflammation. While spatial transcriptomic analysis that combines anatomical information with transcriptomic data of synovial tissue biopsies promises to provide more insights into disease pathogenesis, in vitro functional assays with single-cell resolution will be required to validate current bioinformatic applications. In order to provide a comprehensive approach and translate experimental data to clinical practice, a combination of clinical and molecular data with machine learning has the potential to enhance patient stratification and identify individuals at risk of arthritis that would benefit from early therapeutic intervention. This review aims to provide a comprehensive understanding of the effect of computational approaches in deciphering synovial inflammation pathogenesis and discuss the impact that further experimental and novel computational tools may have on therapeutic target identification and drug development.

A scoping review of current and emerging techniques for evaluation of peripheral nerve health, degeneration and regeneration: part 2, non-invasive imaging

Peripheral neuroregenerative research and therapeutic options are expanding exponentially. With this expansion comes an increasing need to reliably evaluate and quantify nerve health. Valid and responsive measures of the nerve status are essential for both clinical and research purposes for diagnosis, longitudinal follow-up, and monitoring the impact of any intervention. Furthermore, novel biomarkers can elucidate regenerative mechanisms and open new avenues for research. Without such measures, clinical decision-making is impaired, and research becomes more costly, time-consuming, and sometimes infeasible. Part 1 of this two-part scoping review focused on neurophysiology. In part 2, we identify and critically examine many current and emerging non-invasive imaging techniques that have the potential to evaluate peripheral nerve health, particularly from the perspective of regenerative therapies and research.

Automatic classification of Candida species using Raman spectroscopy and machine learning

One of the problems that most affect hospitals is infections by pathogenic microorganisms. Rapid identification and adequate, timely treatment can avoid fatal consequences and the development of antibiotic resistance, so it is crucial to use fast, reliable, and not too laborious techniques to obtain quick results. Raman spectroscopy has proven to be a powerful tool for molecular analysis, meeting these requirements better than traditional techniques. In this work, we have used Raman spectroscopy combined with machine learning algorithms to explore the automatic identification of eleven species of the genus Candida, the most common cause of fungal infections worldwide. The Raman spectra were obtained from more than 220 different measurements of dried drops from pure cultures of each Candida species using a Raman Confocal Microscope with a 532 nm laser excitation source. After developing a spectral preprocessing methodology, a study of the quality and variability of the measured spectra at the isolate and species level, and the spectral features contributing to inter-class variations, showed the potential to discriminate between those pathogenic yeasts. Several machine learning and deep learning algorithms were trained using hyperparameter optimization techniques to find the best possible classifier for this spectral data, in terms of accuracy and lowest possible overfitting. We found that a one-dimensional Convolutional Neural Network (1-D CNN) could achieve above 80 % overall accuracy for the eleven classes spectral dataset, with good generalization capabilities.

A one-dimensional convolutional neural network based deep learning for high accuracy classification of transformation stages in esophageal squamous cell carcinoma tissue using micro-FTIR

Among the most frequently diagnosed cancers in developing countries, esophageal squamous cell carcinoma (ESCC) ranks among the top six causes of death. It would be beneficial if a rapid, accurate, and automatic ESCC diagnostic method could be developed to reduce the workload of pathologists and improve the effectiveness of cancer treatments. Using micro-FTIR spectroscopy, this study classified the transformation stages of ESCC tissues. Based on 6,352 raw micro-FTIR spectra, a one-dimensional convolutional neural network (1D-CNN) model was constructed to classify-five stages. Based on the established model, more than 93% accuracy was achieved at each stage, and the accuracy of identifying proliferation, low grade neoplasia, and ESCC cancer groups was achieved 99% for the test dataset. In this proof-of-concept study, the developed method can be applied to other diseases in order to promote the use of FTIR spectroscopy in cancer pathology.

Recent Trends in SERS-Based Plasmonic Sensors for Disease Diagnostics, Biomolecules Detection, and Machine Learning Techniques

Surface-enhanced Raman spectroscopy/scattering (SERS) has evolved into a popular tool for applications in biology and medicine owing to its ease-of-use, non-destructive, and label-free approach. Advances in plasmonics and instrumentation have enabled the realization of SERS's full potential for the trace detection of biomolecules, disease diagnostics, and monitoring. We provide a brief review on the recent developments in the SERS technique for biosensing applications, with a particular focus on machine learning techniques used for the same. Initially, the article discusses the need for plasmonic sensors in biology and the advantage of SERS over existing techniques. In the later sections, the applications are organized as SERS-based biosensing for disease diagnosis focusing on cancer identification and respiratory diseases, including the recent SARS-CoV-2 detection. We then discuss progress in sensing microorganisms, such as bacteria, with a particular focus on plasmonic sensors for detecting biohazardous materials in view of homeland security. At the end of the article, we focus on machine learning techniques for the (a) identification, (b) classification, and (c) quantification in SERS for biology applications. The review covers the work from 2010 onwards, and the language is simplified to suit the needs of the interdisciplinary audience.

Culture-Independent Raman Spectroscopic Identification of Bacterial Pathogens from Clinical Samples Using Deep Transfer Learning

The rapid identification of bacterial pathogens in clinical samples like blood, urine, pus, and sputum is the need of the hour. Conventional bacterial identification methods like culturing and nucleic acid-based amplification have limitations like poor sensitivity, high cost, slow turnaround time, etc. Raman spectroscopy, a label-free and noninvasive technique, has overcome these drawbacks by providing rapid biochemical signatures from a single bacterium. Raman spectroscopy combined with chemometric methods has been used effectively to identify pathogens. However, a robust approach is needed to utilize Raman features for accurate classification while dealing with complex data sets such as spectra obtained from clinical isolates, showing high sample-to-sample heterogeneity. In this study, we have used Raman spectroscopy-based identification of pathogens from clinical isolates using a deep transfer learning approach at the single-cell level resolution. We have used the data-augmentation method to increase the volume of spectra needed for deep-learning analysis. Our ResNet model could specifically extract the spectral features of eight different pathogenic bacterial species with a 99.99% classification accuracy. The robustness of our model was validated on a set of blinded data sets, a mix of cultured and noncultured bacterial isolates of various origins and types. Our proposed ResNet model efficiently identified the pathogens from the blinded data set with high accuracy, providing a robust and rapid bacterial identification platform for clinical microbiology.

Treatment of chronic liver disease using cell‑engineered constructs: morphofunctional characteristics

Objective: to study the effectiveness of correcting the morphofunctional characteristics of the liver in an experimental model of chronic liver disease (CLD), using implanted cell-engineered constructs (CECs).

Materials and methods. Experiments were carried out on male Wistar rats (n = 80) aged 6–8 months with an initial weight of 230–250 g. CLD was modeled by inoculating the rats with 60% CCl4 oil solution for 42 days based on a modified scheme. Microgel based on recombinant spidroin rS1/9 was used as a matrix for CECs fabrication. Allogeneic liver cells (LCs) and multipotent bone marrow-derived mesenchymal stem cells (BM-MSCs) from a healthy donor were used as the cellular component of the CECs. The effectiveness of the corrective effect of the implanted CECs was assessed in an experimental CLD model (n = 60) in two groups of rats: Group 1 (control, n = 20, 1 mL of saline solution was injected into the damaged liver parenchyma) and Group 2 (experimental, n = 40, CECs containing allogenic LCs and BM-MSCs in a 5 : 1 ratio in a volume of 1 mL were implanted into the damaged liver parenchyma). For long-term monitoring of the CEC state, the CECs were labeled by additional inclusion in Cytodex-3. The effectiveness of the regulatory effect of CECs on regenerative processes in the liver was evaluated using biochemical, morphological and morphometric techniques, as well as by flow cytometry at 90 days after implantation.

Results. In the control group, the mortality rate in CLD was 25%. There was no death in the experimental group with CLD after CEC implantation. The CECs were found to have a corrective effect on the biochemical and morphological parameters of the liver in CLD during 90 days of follow-up, with concomitant preservation of structural cellular homeostasis in the implanted CECs. Conclusion. Implantation of CECs in the liver facilitates effective correction of CLD by activating regenerative processes in the damaged liver, which is due to long-term preservation of structural cellular homeostasis in the CECs.

Deep Learning for Raman Spectroscopy: A Review

Raman spectroscopy (RS) is a spectroscopic method which indirectly measures the vibrational states within samples. This information on vibrational states can be utilized as spectroscopic fingerprints of the sample, which, subsequently, can be used in a wide range of application scenarios to determine the chemical composition of the sample without altering it, or to predict a sample property, such as the disease state of patients. These two examples are only a small portion of the application scenarios, which range from biomedical diagnostics to material science questions. However, the Raman signal is weak and due to the label-free character of RS, the Raman data is untargeted. Therefore, the analysis of Raman spectra is challenging and machine learning based chemometric models are needed. As a subset of representation learning algorithms, deep learning (DL) has had great success in data science for the analysis of Raman spectra and photonic data in general. In this review, recent developments of DL algorithms for Raman spectroscopy and the current challenges in the application of these algorithms will be discussed.

Fast intraoperative histology-based diagnosis of gliomas with third harmonic generation microscopy and deep learning

Management of gliomas requires an invasive treatment strategy, including extensive surgical resection. The objective of the neurosurgeon is to maximize tumor removal while preserving healthy brain tissue. However, the lack of a clear tumor boundary hampers the neurosurgeon's ability to accurately detect and resect infiltrating tumor tissue. Nonlinear multiphoton microscopy, in particular higher harmonic generation, enables label-free imaging of excised brain tissue, revealing histological hallmarks within seconds. Here, we demonstrate a real-time deep learning-based pipeline for automated glioma image analysis, matching video-rate image acquisition. We used a custom noise detection scheme, and a fully-convolutional classification network, to achieve on average 79% binary accuracy, 0.77 AUC and 0.83 mean average precision compared to the consensus of three pathologists, on a preliminary dataset. We conclude that the combination of real-time imaging and image analysis shows great potential for intraoperative assessment of brain tissue during tumor surgery.

Machine Learning of Raman Spectroscopy Data for Classifying Cancers: A Review of the Recent Literature

Raman Spectroscopy has long been anticipated to augment clinical decision making, such as classifying oncological samples. Unfortunately, the complexity of Raman data has thus far inhibited their routine use in clinical settings. Traditional machine learning models have been used to help exploit this information, but recent advances in deep learning have the potential to improve the field. However, there are a number of potential pitfalls with both traditional and deep learning models. We conduct a literature review to ascertain the recent machine learning methods used to classify cancers using Raman spectral data. We find that while deep learning models are popular, and ostensibly outperform traditional learning models, there are many methodological considerations which may be leading to an over-estimation of performance; primarily, small sample sizes which compound sub-optimal choices regarding sampling and validation strategies. Amongst several recommendations is a call to collate large benchmark Raman datasets, similar to those that have helped transform digital pathology, which researchers can use to develop and refine deep learning models.

Noises investigations and image denoising in femtosecond stimulated Raman scattering microscopy

In the literature of SRS microscopy, the hardware characterization usually remains separate from the image processing. In this article, we consider both these aspects and statistical properties analysis of image noise, which plays the vital role of joining links between them. Firstly, we perform hardware characterization by systematic measurements of noise sources, demonstrating that our in-house built microscope is shot noise limited. Secondly, we analyze the statistical properties of the overall image noise, and we prove that the noise distribution can be dependent on image direction, whose origin is the use of a lock-in time constant longer than pixel dwell time. Finally, we compare the performances of two widespread general algorithms, that is, singular value decomposition and discrete wavelet transform, with a method, that is, singular spectrum analysis (SSA), which has been adapted for stimulated Raman scattering images. In order to validate our algorithms, in our investigations lipids droplets have been used and we demonstrate that the adapted SSA method provides an improvement in image denoising.

High frame rate (∼3 Hz) circular photoacoustic tomography using single-element ultrasound transducer aided with deep learning

Significance

In circular scanning photoacoustic tomography (PAT), it takes several minutes to generate an image of acceptable quality, especially with a single-element ultrasound transducer (UST). The imaging speed can be enhanced by faster scanning (with high repetition rate light sources) and using multiple-USTs. However, artifacts arising from the sparse signal acquisition and low signal-to-noise ratio at higher scanning speeds limit the imaging speed. Thus, there is a need to improve the imaging speed of the PAT systems without hampering the quality of the PAT image.

Aim

To improve the frame rate (or imaging speed) of the PAT system by using deep learning (DL).

Approach

For improving the frame rate (or imaging speed) of the PAT system, we propose a novel U-Net-based DL framework to reconstruct PAT images from fast scanning data.

Results

The efficiency of the network was evaluated on both single- and multiple-UST-based PAT systems. Both phantom and in vivo imaging demonstrate that the network can improve the imaging frame rate by approximately sixfold in single-UST-based PAT systems and by approximately twofold in multi-UST-based PAT systems.

Conclusions

We proposed an innovative method to improve the frame rate (or imaging speed) by using DL and with this method, the fastest frame rate of ∼ 3    Hz imaging is achieved without hampering the quality of the reconstructed image.

Towards Raman imaging of centimeter scale tissue areas for real-time opto-molecular visualization of tissue boundaries for clinical applications

Raman spectroscopy combined with augmented reality and mixed reality to reconstruct molecular information of tissue surface.

Benchmarking Spectroscopic Techniques Combined with Machine Learning to Study Oak Barrels for Wine Ageing

Due to its physical, chemical, and structural properties, oakwood is widely used in the production of barrels for wine ageing. When in contact with the wine, oak continuously releases aromatic compounds such as lignin, tannin, and cellulose to the liquid. Due to the release process, oak loses its characteristic aromatic compounds in time; hence, the flavour that it gives to the enclosed wine decreases for repeated wine refills and a barrel replacement is required. Currently, the estimation of the maximum number of refills is empirical and its underestimation or overestimation can impose unnecessary costs and impair the quality of the wine. Therefore, there is a clear need to quantify the presence of the aforementioned aromatic compounds in an oak barrel prior to a refill. This work constitutes a study to examine noninvasive optical biosensing techniques for the characterization of an oak barrel used in wine ageing, towards the development of a model to unveil its lifespan without inducing structural damage. Spectroscopic diagnostic techniques, such as reflectance, fluorescence, and Raman scattering measurements are employed to assess the change in the chemical composition of the oakwood barrel (tannin and lignin presence) and its dependence on repeated refills. To our knowledge, this is the first time that we present a benchmarking study of oak barrel ageing characteristics through spectroscopic methods for the wine industry. The spectroscopic data are processed using standard chemometric techniques, such as Linear Discriminant Analysis and Partial Least Squares Discriminant Analysis. Results of a study of fresh, one-time-used, and two-times-used oak barrel samples demonstrate that reflectance spectroscopy can be a valuable tool for the characterization of oak barrels. Moreover, reflectance spectroscopy has demonstrated the most accurate classification performance. The highest accuracy has been obtained by a Partial Least Squares Discriminant Analysis model that has been able to classify all the oakwood samples from the barrels with >99% accuracy. These preliminary results pave a way for the application of cost-effective and non-invasive biosensing techniques based on reflectance spectroscopy for oak barrels assessment.

Deep learning-based image processing in optical microscopy

Optical microscopy has emerged as a key driver of fundamental research since it provides the ability to probe into imperceptible structures in the biomedical world. For the detailed investigation of samples, a high-resolution image with enhanced contrast and minimal damage is preferred. To achieve this, an automated image analysis method is preferable over manual analysis in terms of both speed of acquisition and reduced error accumulation. In this regard, deep learning (DL)-based image processing can be highly beneficial. The review summarises and critiques the use of DL in image processing for the data collected using various optical microscopic techniques. In tandem with optical microscopy, DL has already found applications in various problems related to image classification and segmentation. It has also performed well in enhancing image resolution in smartphone-based microscopy, which in turn enablse crucial medical assistance in remote places.

Graphical abstract

Attenuated Total Reflection (ATR) Micro-Fourier Transform Infrared (Micro-FT-IR) Spectroscopy to Enhance Repeatability and Reproducibility of Spectra Derived from Single Specimen Organic-Walled Dinoflagellate Cysts

The chemical composition of recent and fossil organic-walled dinoflagellate cyst walls and its diversity is poorly understood and analyses on single microscopic specimens are rare. A series of infrared spectroscopic experiments resulted in the proposition of a standardized attenuated total reflection micro-Fourier transform infrared-based method that allows the collection of robust data sets consisting of spectra from individual dinocysts. These data sets are largely devoid of nonchemical artifacts inherent to other infrared spectrochemical methods, which have typically been used to study similar specimens in the past. The influence of sample preparation, specimen morphology and size and spectral data processing steps is also assessed within this methodological framework. As a result, several guidelines are proposed which facilitate the collection and qualitative interpretation of highly reproducible and repeatable spectrochemical data. These, in turn, pave the way for a systematic exploration of dinocyst chemistry and its assessment as a chemotaxonomical tool or proxy.

RamanNet: a lightweight convolutional neural network for bacterial identification based on Raman spectra

Raman spectroscopy combined convolutional neural network (CNN) enables rapid and accurate identification of the species of bacteria. However, the existing CNN requires a complex hyperparameters model design. Herein, we propose a new simple network architecture with less hyperparameter design and low computation cost, RamanNet, for rapid and accurate identifying of bacteria at the species level based on its Raman spectra. We verified that compared with the previous CNN methods, the RamanNet reached comparable results on the Bacteria-ID Raman spectral dataset and PKU-bacterial Raman spectral datasets, but using only about 1/45 and 1/297 network parameters, respectively. RamanNet achieved an average isolate-level accuracy of 84.7 ± 0.3%, antibiotic treatment identification accuracy of 97.1 ± 0.3%, and distinguished accuracy of 81.6 ± 0.9% for methicillin-resistant and -susceptible Staphylococcus aureus (MRSA and MSSA) on the Bacteria-ID dataset, respectively. Moreover, it achieved an average accuracy of 96.04% on the PKU-bacterial dataset. The RamanNet model benefited from fewer model parameters that can be quickly trained even using CPU. Therefore, our method has the potential to rapidly and accurately identify bacterial species based on their Raman spectra and can be easily extended to other classification tasks based on Raman spectra.

We propose a novel CNN model named RamanNet for rapid and accurate identification of bacteria at the species-level based on Raman spectra. Compared to previous CNN methods, the RamanNet reached comparable results on the Bacteria-ID Raman spectral dataset.

Automated segmentation of metal stent and bioresorbable vascular scaffold in intravascular optical coherence tomography images using deep learning architectures

Percutaneous coronary intervention (PCI) with stent placement is a treatment effective for coronary artery diseases. Intravascular optical coherence tomography (OCT) with high resolution is used clinically to visualize stent deployment and restenosis, facilitating PCI operation and for complication inspection. Automated stent struts segmentation in OCT images is necessary as each pullback of OCT images could contain thousands of stent struts. In this paper, a deep learning framework is proposed and demonstrated for the automated segmentation of two major clinical stent types: metal stents and bioresorbable vascular scaffolds (BVS). U-Net, the current most prominent deep learning network in biomedical segmentation, was implemented for segmentation with cropped input. The architectures of MobileNetV2 and DenseNet121 were also adapted into U-Net for improvement in speed and accuracy. The results suggested that the proposed automated algorithm's segmentation performance approaches the level of independent human obsevers and is feasible for both types of stents despite their distinct appearance. U-Net with DenseNet121 encoder (U-Dense) performed best with Dice's coefficient of 0.86 for BVS segmentation, and precision/recall of 0.92/0.92 for metal stent segmentation under optimal crop window size of 256.

High-Throughput Molecular Imaging via Deep-Learning-Enabled Raman Spectroscopy

Raman spectroscopy enables nondestructive, label-free imaging with unprecedented molecular contrast, but is limited by slow data acquisition, largely preventing high-throughput imaging applications. Here, we present a comprehensive framework for higher-throughput molecular imaging via deep-learning-enabled Raman spectroscopy, termed DeepeR, trained on a large data set of hyperspectral Raman images, with over 1.5 million spectra (400 h of acquisition) in total. We first perform denoising and reconstruction of low signal-to-noise ratio Raman molecular signatures via deep learning, with a 10× improvement in the mean-squared error over common Raman filtering methods. Next, we develop a neural network for robust 2-4× spatial super-resolution of hyperspectral Raman images that preserve molecular cellular information. Combining these approaches, we achieve Raman imaging speed-ups of up to 40-90×, enabling good-quality cellular imaging with a high-resolution, high signal-to-noise ratio in under 1 min. We further demonstrate Raman imaging speed-up of 160×, useful for lower resolution imaging applications such as the rapid screening of large areas or for spectral pathology. Finally, transfer learning is applied to extend DeepeR from cell to tissue-scale imaging. DeepeR provides a foundation that will enable a host of higher-throughput Raman spectroscopy and molecular imaging applications across biomedicine.

Infrared nanoscopy and tomography of intracellular structures

Although techniques such as fluorescence-based super-resolution imaging or confocal microscopy simultaneously gather both morphological and chemical data, these techniques often rely on the use of localized and chemically specific markers. To eliminate this flaw, we have developed a method of examining cellular cross sections using the imaging power of scattering-type scanning near-field optical microscopy and Fourier-transform infrared spectroscopy at a spatial resolution far beyond the diffraction limit. Herewith, nanoscale surface and volumetric chemical imaging is performed using the intrinsic contrast generated by the characteristic absorption of mid-infrared radiation by the covalent bonds. We employ infrared nanoscopy to study the subcellular structures of eukaryotic (Chlamydomonas reinhardtii) and prokaryotic (Escherichia coli) species, revealing chemically distinct regions within each cell such as the microtubular structure of the flagellum. Serial 100 nm-thick cellular cross-sections were compiled into a tomogram yielding a three-dimensional infrared image of subcellular structure distribution at 20 nm resolution. The presented methodology is able to image biological samples complementing current fluorescence nanoscopy but at less interference due to the low energy of infrared radiation and the absence of labeling.

Deep learning-based classification of blue light cystoscopy imaging during transurethral resection of bladder tumors

Bladder cancer is one of the top 10 frequently occurring cancers and leads to most cancer deaths worldwide. Recently, blue light (BL) cystoscopy-based photodynamic diagnosis was introduced as a unique technology to enhance the detection of bladder cancer, particularly for the detection of flat and small lesions. Here, we aim to demonstrate a BL image-based artificial intelligence (AI) diagnostic platform using 216 BL images, that were acquired in four different urological departments and pathologically identified with respect to cancer malignancy, invasiveness, and grading. Thereafter, four pre-trained convolution neural networks were utilized to predict image malignancy, invasiveness, and grading. The results indicated that the classification sensitivity and specificity of malignant lesions are 95.77% and 87.84%, while the mean sensitivity and mean specificity of tumor invasiveness are 88% and 96.56%, respectively. This small multicenter clinical study clearly shows the potential of AI based classification of BL images allowing for better treatment decisions and potentially higher detection rates.

Sounding out the hidden data: A concise review of deep learning in photoacoustic imaging

The rapidly evolving field of photoacoustic tomography utilizes endogenous chromophores to extract both functional and structural information from deep within tissues. It is this power to perform precise quantitative measurements in vivo-with endogenous or exogenous contrast-that makes photoacoustic tomography highly promising for clinical translation in functional brain imaging, early cancer detection, real-time surgical guidance, and the visualization of dynamic drug responses. Considering photoacoustic tomography has benefited from numerous engineering innovations, it is of no surprise that many of photoacoustic tomography's current cutting-edge developments incorporate advances from the equally novel field of artificial intelligence. More specifically, alongside the growth and prevalence of graphical processing unit capabilities within recent years has emerged an offshoot of artificial intelligence known as deep learning. Rooted in the solid foundation of signal processing, deep learning typically utilizes a method of optimization known as gradient descent to minimize a loss function and update model parameters. There are already a number of innovative efforts in photoacoustic tomography utilizing deep learning techniques for a variety of purposes, including resolution enhancement, reconstruction artifact removal, undersampling correction, and improved quantification. Most of these efforts have proven to be highly promising in addressing long-standing technical obstacles where traditional solutions either completely fail or make only incremental progress. This concise review focuses on the history of applied artificial intelligence in photoacoustic tomography, presents recent advances at this multifaceted intersection of fields, and outlines the most exciting advances that will likely propagate into promising future innovations.

Perspective on fast-evolving photoacoustic tomography

SIGNIFICANCE

Acoustically detecting the rich optical absorption contrast in biological tissues, photoacoustic tomography (PAT) seamlessly bridges the functional and molecular sensitivity of optical excitation with the deep penetration and high scalability of ultrasound detection. As a result of continuous technological innovations and commercial development, PAT has been playing an increasingly important role in life sciences and patient care, including functional brain imaging, smart drug delivery, early cancer diagnosis, and interventional therapy guidance.

AIM

Built on our 2016 tutorial article that focused on the principles and implementations of PAT, this perspective aims to provide an update on the exciting technical advances in PAT.

APPROACH

This perspective focuses on the recent PAT innovations in volumetric deep-tissue imaging, high-speed wide-field microscopic imaging, high-sensitivity optical ultrasound detection, and machine-learning enhanced image reconstruction and data processing. Representative applications are introduced to demonstrate these enabling technical breakthroughs in biomedical research.

CONCLUSIONS

We conclude the perspective by discussing the future development of PAT technologies.

Diffuse reflectance and machine learning techniques to differentiate colorectal cancer ex vivo

In this study, we used machine learning techniques to reconstruct the wavelength dependence of the absorption coefficient of human normal and pathological colorectal mucosa tissues. Using only diffuse reflectance spectra from the ex vivo mucosa tissues as input to algorithms, several approaches were tried before obtaining good matching between the generated absorption coefficients and the ones previously calculated for the mucosa tissues from invasive experimental spectral measurements. Considering the optimized match for the results generated with the multilayer perceptron regression method, we were able to identify differentiated accumulation of lipofuscin in the absorption coefficient spectra of both mucosa tissues as we have done before with the corresponding results calculated directly from invasive measurements. Considering the random forest regressor algorithm, the estimated absorption coefficient spectra almost matched the ones previously calculated. By subtracting the absorption of lipofuscin from these spectra, we obtained similar hemoglobin ratios at 410/550 nm: 18.9-fold/9.3-fold for the healthy mucosa and 46.6-fold/24.2-fold for the pathological mucosa, while from direct calculations, those ratios were 19.7-fold/10.1-fold for the healthy mucosa and 33.1-fold/17.3-fold for the pathological mucosa. The higher values obtained in this study indicate a higher blood content in the pathological samples used to measure the diffuse reflectance spectra. In light of such accuracy and sensibility to the presence of hidden absorbers, with a different accumulation between healthy and pathological tissues, good perspectives become available to develop minimally invasive spectroscopy methods for in vivo early detection and monitoring of colorectal cancer.

Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning

Hematoxylin and Eosin (H&E) staining is the 'gold-standard' method in histopathology. However, standard H&E staining of high-quality tissue sections requires long sample preparation times including sample embedding, which restricts its application for 'real-time' disease diagnosis. Due to this reason, a label-free alternative technique like non-linear multimodal (NLM) imaging, which is the combination of three non-linear optical modalities including coherent anti-Stokes Raman scattering, two-photon excitation fluorescence and second-harmonic generation, is proposed in this work. To correlate the information of the NLM images with H&E images, this work proposes computational staining of NLM images using deep learning models in a supervised and an unsupervised approach. In the supervised and the unsupervised approach, conditional generative adversarial networks (CGANs) and cycle conditional generative adversarial networks (cycle CGANs) are used, respectively. Both CGAN and cycle CGAN models generate pseudo H&E images, which are quantitatively analyzed based on mean squared error, structure similarity index and color shading similarity index. The mean of the three metrics calculated for the computationally generated H&E images indicate significant performance. Thus, utilizing CGAN and cycle CGAN models for computational staining is beneficial for diagnostic applications without performing a laboratory-based staining procedure. To the author's best knowledge, it is the first time that NLM images are computationally stained to H&E images using GANs in an unsupervised manner.

Disease-Relevant Single Cell Photonic Signatures Identify S100β Stem Cells and their Myogenic Progeny in Vascular Lesions

A hallmark of subclinical atherosclerosis is the accumulation of vascular smooth muscle cell (SMC)-like cells leading to intimal thickening and lesion formation. While medial SMCs contribute to vascular lesions, the involvement of resident vascular stem cells (vSCs) remains unclear. We evaluated single cell photonics as a discriminator of cell phenotype in vitro before the presence of vSC within vascular lesions was assessed ex vivo using supervised machine learning and further validated using lineage tracing analysis. Using a novel lab-on-a-Disk(Load) platform, label-free single cell photonic emissions from normal and injured vessels ex vivo were interrogated and compared to freshly isolated aortic SMCs, cultured Movas SMCs, macrophages, B-cells, S100β⁺ mVSc, bone marrow derived mesenchymal stem cells (MSC) and their respective myogenic progeny across five broadband light wavelengths (λ465 - λ670 ± 20 nm). We found that profiles were of sufficient coverage, specificity, and quality to clearly distinguish medial SMCs from different vascular beds (carotid vs aorta), discriminate normal carotid medial SMCs from lesional SMC-like cells ex vivo following flow restriction, and identify SMC differentiation of a series of multipotent stem cells following treatment with transforming growth factor beta 1 (TGF- β1), the Notch ligand Jagged1, and Sonic Hedgehog using multivariate analysis, in part, due to photonic emissions from enhanced collagen III and elastin expression. Supervised machine learning supported genetic lineage tracing analysis of S100β⁺ vSCs and identified the presence of S100β⁺vSC-derived myogenic progeny within vascular lesions. We conclude disease-relevant photonic signatures may have predictive value for vascular disease.

A representation learning approach for recovering scatter-corrected spectra from Fourier-transform infrared spectra of tissue samples

Infrared spectra obtained from cell or tissue specimen have commonly been observed to involve a significant degree of scattering effects, often Mie scattering, which probably overshadows biochemically relevant spectral information by a nonlinear, nonadditive spectral component in Fourier transform infrared (FTIR) spectroscopic measurements. Correspondingly, many successful machine learning approaches for FTIR spectra have relied on preprocessing procedures that computationally remove the scattering components from an infrared spectrum. We propose an approach to approximate this complex preprocessing function using deep neural networks. As we demonstrate, the resulting model is not just several orders of magnitudes faster, which is important for real-time clinical applications, but also generalizes strongly across different tissue types. Using Bayesian machine learning approaches, our approach unveils model uncertainty that coincides with a band shift in the amide I region that occurs when scattering is removed computationally based on an established physical model. Furthermore, our proposed method overcomes the trade-off between computation time and the corrected spectrum being biased towards an artificial reference spectrum.

Deep Learning for Biospectroscopy and Biospectral Imaging: State-of-the-Art and Perspectives

With the advances in instrumentation and sampling techniques, there is an explosive growth of data from molecular and cellular samples. The call to extract more information from the large data sets has greatly challenged the conventional chemometrics method. Deep learning, which utilizes very large data sets for finding hidden features therein and for making accurate predictions for a wide range of applications, has been applied in an unbelievable pace in biospectroscopy and biospectral imaging in the recent 3 years. In this Feature, we first introduce the background and basic knowledge of deep learning. We then focus on the emerging applications of deep learning in the data preprocessing, feature detection, and modeling of the biological samples for spectral analysis and spectroscopic imaging. Finally, we highlight the challenges and limitations in deep learning and the outlook for future directions.

LV-GAN: A deep learning approach for limited-view optoacoustic imaging based on hybrid datasets

The optoacoustic imaging (OAI) methods are rapidly evolving for resolving optical contrast in medical imaging applications. In practice, measurement strategies are commonly implemented under limited-view conditions due to oversized image objectives or system design limitations. Data acquired by limited-view detection may impart artifacts and distortions in reconstructed optoacoustic (OA) images. We propose a hybrid data-driven deep learning approach based on generative adversarial network (GAN), termed as LV-GAN, to efficiently recover high quality images from limited-view OA images. Trained on both simulation and experiment data, LV-GAN is found capable of achieving high recovery accuracy even under limited detection angles less than 60^° . The feasibility of LV-GAN for artifact removal in biological applications was validated by ex vivo experiments based on two different OAI systems, suggesting high potential of a ubiquitous use of LV-GAN to optimize image quality or system design for different scanners and application scenarios.

Toward rapid infectious disease diagnosis with advances in surface-enhanced Raman spectroscopy

In a pandemic era, rapid infectious disease diagnosis is essential. Surface-enhanced Raman spectroscopy (SERS) promises sensitive and specific diagnosis including rapid point-of-care detection and drug susceptibility testing. SERS utilizes inelastic light scattering arising from the interaction of incident photons with molecular vibrations, enhanced by orders of magnitude with resonant metallic or dielectric nanostructures. While SERS provides a spectral fingerprint of the sample, clinical translation is lagged due to challenges in consistency of spectral enhancement, complexity in spectral interpretation, insufficient specificity and sensitivity, and inefficient workflow from patient sample collection to spectral acquisition. Here, we highlight the recent, complementary advances that address these shortcomings, including (1) design of label-free SERS substrates and data processing algorithms that improve spectral signal and interpretability, essential for broad pathogen screening assays; (2) development of new capture and affinity agents, such as aptamers and polymers, critical for determining the presence or absence of particular pathogens; and (3) microfluidic and bioprinting platforms for efficient clinical sample processing. We also describe the development of low-cost, point-of-care, optical SERS hardware. Our paper focuses on SERS for viral and bacterial detection, in hopes of accelerating infectious disease diagnosis, monitoring, and vaccine development. With advances in SERS substrates, machine learning, and microfluidics and bioprinting, the specificity, sensitivity, and speed of SERS can be readily translated from laboratory bench to patient bedside, accelerating point-of-care diagnosis, personalized medicine, and precision health.

Another decade of photoacoustic imaging

Photoacoustic imaging - a hybrid biomedical imaging modality finding its way to clinical practices. Although the photoacoustic phenomenon was known more than a century back, only in the last two decades it has been widely researched and used for biomedical imaging applications. In this review we focus on the development and progress of the technology in the last decade (2010-2020). From becoming more and more user friendly, cheaper in cost, portable in size, photoacoustic imaging promises a wide range of applications, if translated to clinic. The growth of photoacoustic community is steady, and with several new directions researchers are exploring, it is inevitable that photoacoustic imaging will one day establish itself as a regular imaging system in the clinical practices.

SSNOMBACTER: A collection of scattering-type scanning near-field optical microscopy and atomic force microscopy images of bacterial cells

BACKGROUND

In recent years, a variety of imaging techniques operating at nanoscale resolution have been reported. These techniques have the potential to enrich our understanding of bacterial species relevant to human health, such as antibiotic-resistant pathogens. However, owing to the novelty of these techniques, their use is still confined to addressing very particular applications, and their availability is limited owing to associated costs and required expertise. Among these, scattering-type scanning near field optical microscopy (s-SNOM) has been demonstrated as a powerful tool for exploring important optical properties at nanoscale resolution, depending only on the size of a sharp tip. Despite its huge potential to resolve aspects that cannot be tackled otherwise, the penetration of s-SNOM into the life sciences is still proceeding at a slow pace for the aforementioned reasons.

RESULTS

In this work we introduce SSNOMBACTER, a set of s-SNOM images collected on 15 bacterial species. These come accompanied by registered Atomic Force Microscopy images, which are useful for placing nanoscale optical information in a relevant topographic context.

CONCLUSIONS

The proposed dataset aims to augment the popularity of s-SNOM and for accelerating its penetration in life sciences. Furthermore, we consider this dataset to be useful for the development and benchmarking of image analysis tools dedicated to s-SNOM imaging, which are scarce, despite the high need. In this latter context we discuss a series of image processing and analysis applications where SSNOMBACTER could be of help.