Multispectral detection of dietary fiber content in Chinese cabbage leaves across different growth periods

Multispectral imaging, combined with stoichiometric values, was used to construct a prediction model to measure changes in dietary fiber (DF) content in Chinese cabbage leaves across different growth periods. Based on all the spectral bands (365-970 nm) and characteristic spectral bands (430, 880, 590, 490, 690 nm), eight quantitative prediction models were established using four machine learning algorithms, namely random forest (RF), backpropagation neural network, radial basis function, and multiple linear regression. Finally, a quantitative prediction model of RF learning algorithm is constructed based on all spectral bands, which has good prediction accuracy and model robustness, prediction performance with R2 of 0.9023, root mean square error (RMSE) of 2.7182 g/100 g, residual predictive deviation (RPD) of 3.1220 > 3.0. In summary, this model efficiently detects changes in DF content across different growth periods of Chinese cabbage, which offers technical support for vegetable sorting and grading in the field.

Hyperspectral and multispectral imaging in neurosurgery: a systematic literature review and meta-analysis

INTRODUCTION

The neuro-surgical community is witnessing a rising interest for surgical application of multispectral/hyperspectral imaging. Several potential technical applications of this optical imaging are reported, but the set-up is variable and so are the processing methods. We present a systematic review of the relevant literature on the topic.

MATERIALS AND METHODS

A literature search based on the PRISMA principles was performed on PubMed, SCOPUS, and Web of Science, using MESH terms and Boolean operators. Papers regarding intra-operative in-vivo application of multispectral and/or hyperspectral imaging in humans during neurosurgical procedures were included. Papers reporting technologies related to radiological applications were excluded. A meta-analysis on the performance metrics was also conducted.

RESULTS

Our search string retrieved 20 papers. The main applications of optical imaging during neurosurgery concern tumour detection and improvement of the extent of resection (15 papers) or visualization of perfusion changes during neuro-oncology or neuro-vascular surgery (5 papers). All the retrieved articles were pilot studies, proof of concepts, or case reports, with limited number of patients recruited. Sensitivity, specificity, and accuracy were promising in most of the reports, but the metanalysis showed heterogeneous approaches and results among studies.

CONCLUSIONS

The present review shows that several approaches are currently being tested to integrate hyperspectral imaging in neurosurgery, but most of the studies reported a limited pool of patients, with different approaches to data collection and analysis. Further studies on larger cohorts of patients are therefore desirable to fully explore the potential of this imaging technique.

Feasibility of intra-operative image guidance in burn excision surgery with multispectral imaging and deep learning

BACKGROUND

Exposing a healthy wound bed for skin grafting is an important step during burn surgery to ensure graft take and maintain good functional outcomes. Currently, the removal of non-viable tissue in the burn wound bed during excision is determined by expert clinician judgment. Using a porcine model of tangential burn excision, we investigated the effectiveness of an intraoperative multispectral imaging device combined with artificial intelligence to aid clinician judgment for the excision of non-viable tissue.

METHODS

Multispectral imaging data was obtained from serial tangential excisions of thermal burn injuries and used to train a deep learning algorithm to identify the presence and location of non-viable tissue in the wound bed. Following algorithm development, we studied the ability of two surgeons to estimate wound bed viability, both unaided and aided by the imaging device.

RESULTS

The deep learning algorithm was 87% accurate in identifying the viability of a burn wound bed. When paired with the surgeons, this device significantly improved their abilities to determine the viability of the wound bed by 25% (p = 0.03). Each time a surgeon changed their decision after seeing the AI model output, it was always a change from an incorrect decision to excise more tissue to a correct decision to stop excision.

CONCLUSION

This study provides insight into the feasibility of image-guided burn excision, its effect on surgeon decision making, and suggests further investigation of a real-time imaging system for burn surgery could reduce over-excision of burn wounds.

Heterogeneity in intrahepatic macrophage populations and druggable target expression in patients with steatotic liver disease-related fibrosis

Background & Aims

Clinical trials for reducing fibrosis in steatotic liver disease (SLD) have targeted macrophages with variable results. We evaluated intrahepatic macrophages in patients with SLD to determine if activity scores or fibrosis stages influenced phenotypes and expression of druggable targets, such as CCR2 and galectin-3.

Methods

Liver biopsies from controls or patients with minimal or advanced fibrosis were subject to gene expression analysis using nCounter to determine differences in macrophage-related genes (n = 30). To investigate variability among individual patients, we compared additional biopsies by staining them with multiplex antibody panels (CD68/CD14/CD16/CD163/Mac387 or CD163/CCR2/galectin-3/Mac387) followed by spectral imaging and spatial analysis. Algorithms that utilize deep learning/artificial intelligence were applied to create cell cluster plots, phenotype profile maps, and to determine levels of protein expression (n = 34).

Results

Several genes known to be pro-fibrotic (e.g. CD206, TREM2, CD163, and ARG1) showed either no significant differences or significantly decreased with advanced fibrosis. Although marked variability in gene expression was observed in individual patients with cirrhosis, several druggable targets and their ligands (e.g. CCR2, CCR5, CCL2, CCL5, and LGALS3) were significantly increased when compared to patients with minimal fibrosis. Antibody panels identified populations that were significantly increased (e.g. Mac387+), decreased (e.g. CD14+), or enriched (e.g. interactions of Mac387) in patients that had progression of disease or advanced fibrosis. Despite heterogeneity in patients with SLD, several macrophage phenotypes and druggable targets showed a positive correlation with increasing NAFLD activity scores and fibrosis stages.

Conclusions

Patients with SLD have markedly varied macrophage- and druggable target-related gene and protein expression in their livers. Several patients had relatively high expression, while others were like controls. Overall, patients with more advanced disease had significantly higher expression of CCR2 and galectin-3 at both the gene and protein levels.

Impact and implications

Appreciating individual differences within the hepatic microenvironment of patients with SLD may be paramount to developing effective treatments. These results may explain why such a small percentage of patients have responded to macrophage-targeting therapies and provide additional support for precision medicine-guided treatment of chronic liver diseases.

Multispectral-derived genotypic similarities from budget cameras allow grain yield prediction and genomic selection augmentation in single and multi-environment scenarios in spring wheat

With abundant available genomic data, genomic selection has become routine in many plant breeding programs. Multispectral data captured by UAVs showed potential for grain yield (GY) prediction in many plant species using machine learning; however, the possibilities of utilizing this data to augment genomic prediction models still need to be explored. We collected high-throughput phenotyping (HTP) multispectral data in a genotyped multi-environment large-scale field trial using two cost-effective cameras to fill this gap. We tested back to back the prediction ability of GY prediction models, including genomic (G matrix), multispectral-derived (M matrix), and environmental (E matrix) relationships using best linear unbiased predictor (BLUP) methodology in single and multi-environment scenarios. We discovered that M allows for GY prediction comparable to the G matrix and that models using both G and M matrices show superior accuracies and errors compared with G or M alone, both in single and multi-environment scenarios. We showed that the M matrix is not entirely environment-specific, and the genotypic relationships become more robust with more data capture sessions over the season. We discovered that the optimal time for data capture occurs during grain filling and that camera bands with the highest heritability are important for GY prediction using the M matrix. We showcased that GY prediction can be performed using only an RGB camera, and even a single data capture session can yield valuable data for GY prediction. This study contributes to a better understanding of multispectral data and its relationships. It provides a flexible framework for improving GS protocols without significant investments or software customization.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11032-024-01449-w.

Spectral Reflectance Measurements

This chapter provides a methodology for evaluating plant health and leaf characteristics using spectral reflectance. It provides a step-by-step guide to using spectrometers for high-resolution point measurements of leaf spectral reflectance and multispectral imaging for capturing spatial data, emphasizing the importance of consistent measurement conditions. The chapter further explores the intricacies of multispectral imaging, including calibration, data collection, and image processing. Finally, this chapter delves into the application of various spectral indices for the quantification of key traits such as pigment content, the status of the xanthophyll cycle, water content, and how to identify spectral regions of interest for further research and development. Serving as a guide for researchers and practitioners in plant science, this chapter provides a straightforward framework for plant health assessment using spectral reflectance.

P-MIRU, a Polarized Multispectral Imaging System, Reveals Reflection Information on the Biological Surface

Reflection light forms the core of our visual perception of the world. We can obtain vast information by examining reflection light from biological surfaces, including pigment composition and distribution, tissue structure and surface microstructure. However, because of the limitations in our visual system, the complete information in reflection light, which we term 'reflectome', cannot be fully exploited. For example, we may miss reflection light information outside our visible wavelengths. In addition, unlike insects, we have virtually no sensitivity to light polarization. We can detect non-chromatic information lurking in reflection light only with appropriate devices. Although previous studies have designed and developed systems for specialized uses supporting our visual systems, we still do not have a versatile, rapid, convenient and affordable system for analyzing broad aspects of reflection from biological surfaces. To overcome this situation, we developed P-MIRU, a novel multispectral and polarization imaging system for reflecting light from biological surfaces. The hardware and software of P-MIRU are open source and customizable and thus can be applied for virtually any research on biological surfaces. Furthermore, P-MIRU is a user-friendly system for biologists with no specialized programming or engineering knowledge. P-MIRU successfully visualized multispectral reflection in visible/non-visible wavelengths and simultaneously detected various surface phenotypes of spectral polarization. The P-MIRU system extends our visual ability and unveils information on biological surfaces.

Multispectral imaging: Review of current applications

Multispectral imaging (MSI) is a unique layer-by-layer imaging technique that allows the visualization of a wide array of retinal and choroidal pathologies including retinovascular disorders, retinal pigment epithelial changes, and choroidal lesions. Herein, we summarize the basic imaging principles and current applications of MSI together with recent technology advances in the field. MSI detects reflectance signal from both normal chorioretinal tissue and pathological lesions. Either hyperreflectance or hyporeflectance reveals the absorption activity of pigments such as hemoglobin and melanin and the reflection from interfaces such as the posterior hyaloid. Advances in MSI technique include creation of a retinal and choroidal oxy-deoxy map that could provide a better understanding of blood oxygen saturation within lesions as well as better interpretation of reflectance phenomenon of MSI images such as the different reflectance from the Sattler and Haller layers described in this review.

Multispectral Imaging Method for Rapid Identification and Analysis of Paraffin-Embedded Pathological Tissues

The study of the interaction between light and biological tissue is of great help in the identification of diseases as well as structural alterations in tissues. In the present study, we have developed a tissue diagnostic technique by using multispectral imaging in the visible spectrum combined with principal component analysis (PCA). We used information from the propagation of light through paraffin-embedded tissues to assess differences in the eye tissues of control mouse embryos compared to mouse embryos whose mothers were deprived of folic acid (FA), a crucial vitamin necessary for the growth and development of the fetus. After acquiring the endmembers from the multispectral images, spectral unmixing was used to identify the abundances of those endmembers in each pixel. For each acquired image, the final analysis was performed by performing a pixel-by-pixel and wavelength-by-wavelength absorbance calculation. Non-negative least squares (NNLS) were used in this research. The abundance maps obtained for the first endmember revealed vascular alterations (vitreous and choroid) in the embryos with maternal FA deficiency. However, the abundance maps obtained for the third endmember showed alterations in the texture of some tissues such as the lens and retina. Results indicated that multispectral imaging applied to paraffin-embedded tissues enhanced tissue visualization. Using this method, first, it can be seen tissue damage location and then decide what kind of biological techniques to apply.

Dual-wavelength UV-visible metalens for multispectral photoacoustic microscopy: A simulation study

Photoacoustic microscopy is advancing with research on utilizing ultraviolet and visible light. Dual-wavelength approaches are sought for observing DNA/RNA- and vascular-related disorders. However, the availability of high numerical aperture lenses covering both ultraviolet and visible wavelengths is severely limited due to challenges such as chromatic aberration in the optics. Herein, we present a groundbreaking proposal as a pioneering simulation study for incorporating multilayer metalenses into ultraviolet-visible photoacoustic microscopy. The proposed metalens has a thickness of 1.4 µm and high numerical aperture of 0.8. By arranging cylindrical hafnium oxide nanopillars, we design an achromatic transmissive lens for 266 and 532 nm wavelengths. The metalens achieves a diffraction-limited focal spot, surpassing commercially available objective lenses. Through three-dimensional photoacoustic simulation, we demonstrate high-resolution imaging with superior endogenous contrast of targets with ultraviolet and visible optical absorption bands. This metalens will open new possibilities for downsized multispectral photoacoustic microscopy in clinical and preclinical applications.

Comparison of deep learning models for digital H&E staining from unpaired label-free multispectral microscopy images

BACKGROUND AND OBJECTIVE

This paper presents the quantitative comparison of three generative models of digital staining, also known as virtual staining, in H&E modality (i.e., Hematoxylin and Eosin) that are applied to 5 types of breast tissue. Moreover, a qualitative evaluation of the results achieved with the best model was carried out. This process is based on images of samples without staining captured by a multispectral microscope with previous dimensional reduction to three channels in the RGB range.

METHODS

The models compared are based on conditional GAN (pix2pix) which uses images aligned with/without staining, and two models that do not require image alignment, Cycle GAN (cycleGAN) and contrastive learning-based model (CUT). These models are compared based on the structural similarity and chromatic discrepancy between samples with chemical staining and their corresponding ones with digital staining. The correspondence between images is achieved after the chemical staining images are subjected to digital unstaining by means of a model obtained to guarantee the cyclic consistency of the generative models.

RESULTS

The comparison of the three models corroborates the visual evaluation of the results showing the superiority of cycleGAN both for its larger structural similarity with respect to chemical staining (mean value of SSIM ∼ 0.95) and lower chromatic discrepancy (10%). To this end, quantization and calculation of EMD (Earth Mover's Distance) between clusters is used. In addition, quality evaluation through subjective psychophysical tests with three experts was carried out to evaluate quality of the results with the best model (cycleGAN).

CONCLUSIONS

The results can be satisfactorily evaluated by metrics that use as reference image a chemically stained sample and the digital staining images of the reference sample with prior digital unstaining. These metrics demonstrate that generative staining models that guarantee cyclic consistency provide the closest results to chemical H&E staining that also is consistent with the result of qualitative evaluation by experts.

Detecting and distinguishing between apicultural plants using UAV multispectral imaging

Detecting and distinguishing apicultural plants are important elements of the evaluation and quantification of potential honey production worldwide. Today, remote sensing can provide accurate plant distribution maps using rapid and efficient techniques. In the present study, a five-band multispectral unmanned aerial vehicle (UAV) was used in an established beekeeping area on Lemnos Island, Greece, for the collection of high-resolution images from three areas where Thymus capitatus and Sarcopoterium spinosum are present. Orthophotos of UAV bands for each area were used in combination with vegetation indices in the Google Earth Engine (GEE) platform, to classify the area occupied by the two plant species. From the five classifiers (Random Forest, RF; Gradient Tree Boost, GTB; Classification and Regression Trees, CART; Mahalanobis Minimum Distance, MMD; Support Vector Machine, SVM) in GEE, the RF gave the highest overall accuracy with a Kappa coefficient reaching 93.6%, 98.3%, 94.7%, and coefficient of 0.90, 0.97, 0.92 respectively for each case study. The training method used in the present study detected and distinguish the two plants with great accuracy and results were confirmed using 70% of the total score to train the GEE and 30% to assess the method's accuracy. Based on this study, identification and mapping of Thymus capitatus areas is possible and could help in the promotion and protection of this valuable species which, on many Greek Islands, is the sole foraging plant of honeybees.

Galactosidase-catalyzed fluorescence amplification method (GAFAM): sensitive fluorescent immunohistochemistry using novel fluorogenic β-galactosidase substrates and its application in multiplex immunostaining

Multiplex immunohistochemistry/multiplex immunofluorescence (mIHC/mIF) enables the simultaneous detection of multiple markers in a single tissue section by visualizing the markers in different colors. Currently, tyramide signal amplification (TSA) is the most commonly used method because it is heat resistant to multiplexing. SPiDER-βGal (6'-(diethylamino)-4'-(fluoromethyl)spiro[isobenzofuran-1(3H),9'-[9H]xanthen]-3'-yl β-D-galactopyranoside), a novel fluorogenic substrate of β-galactosidase (β-gal) was reported recently. Its properties are favorable for application in sensitive mIF based on quinone methide chemistry. Combining SPiDER-βGal with its related substrates, a novel, sensitive fluorescent IHC method for formalin-fixed paraffin-embedded (FFPE) sections was developed, named the galactosidase-catalyzed fluorescence amplification method (GAFAM). Evaluation of GAFAM indicated the following characteristics: (1) the entire GAFAM procedure was complete within a few hours; (2) the optimal working concentration of the substrates was 20 μM; (3) the fluorescent product was heat resistant; (4) the GAFAM exhibited sensitivity comparable with that of TSA, which was higher than that of conventional IF; and (5) the GAFAM was applicable to mIF and multispectral imaging. GAFAM is expected to be applicable to IF (or mIF in combination with TSA), and is a promising tool for facilitating morphological research in various fields of life science.

MAdCAM-1 does not play a central role in the early pathophysiology of autoimmune hepatitis

INTRODUCTION

CD4+ T cells are thought to have a central role in the pathogenesis of autoimmune hepatitis (AIH). Mucosal addressin cell adhesion molecule-1 (MAdCAM-1) directs homing of CD4+ T cells in the alimentary tract and is a therapeutic target in inflammatory bowel diseases. Here we assessed MAdCAM-1 expression in AIH and viral hepatitis and related its expression with immune infiltrate analysis and histopathological key features.

METHODS

Hepatic portal areas of pretreatment biopsies (n=10) and follow-up biopsies (n=9) of patients with a confirmed diagnosis of AIH were assessed for MAdCAM-1 expression and infiltrate composition using immunohistochemistry and multispectral imaging (Vectra® Polaris™). Controls consisted of biopsies of patients with untreated chronic viral hepatitis B or C (n=22).

RESULTS

MAdCAM-1 expression on endothelium was sparsely present in portal fields of two treatment-naïve AIH patients. Three patients showed MAdCAM-1 expression within lymphoid aggregates. No expression of significance (including single-cell expression) was observed in the remaining 6 patients. In contrast, viral hepatitis C biopsies showed endothelial MAdCAM-1 expression in 8 of 13 untreated patients. Densities of both B-cells (CD20+) and CD4+ T-cells (CD3+ CD8-) were increased in AIH and viral hepatitis patients with MAdCAM-1 expression.

CONCLUSION

MAdCAM-1 was detected in liver biopsies in a minority of patients with AIH at the time of diagnosis suggesting no central role in its pathophysiology. Lymphoid or reticular MAdCAM-1 pattern expression was associated with more dense infiltrates of both B-cells and CD4⁺ T-cells, and may be related to the formation of secondary lymphoid follicles.

Characterization of the immune infiltrate in mouse tissue by multiplex immunofluorescence

Multiplexed immunofluorescence imaging of formalin-fixed, paraffin-embedded (FFPE) specimens mounted on glass slides allow the identification of multiple cell phenotypes while retaining spatial and morphological context. Multiplex immunofluorescence protocols have already been developed and validated for mouse tissues. Immunophenotyping analysis reliably depicts the immune landscape of cancer tissues that has been demonstrated to influence cancer development and progression as well as to have an impact on therapy responsiveness and resistance. Here, we describe a method for multiplexed fluorescence image analysis, enabling analysis of mouse cancer morphology and cell phenotypes in FFPE sections.

SUFI: an automated approach to spectral unmixing of fluorescent multiplex images captured in mouse and post-mortem human brain tissues

BACKGROUND

Multispectral fluorescence imaging coupled with linear unmixing is a form of image data collection and analysis that allows for measuring multiple molecular signals in a single biological sample. Multiple fluorescent dyes, each measuring a unique molecule, are simultaneously measured and subsequently "unmixed" to provide a read-out for each molecular signal. This strategy allows for measuring highly multiplexed signals in a single data capture session, such as multiple proteins or RNAs in tissue slices or cultured cells, but can often result in mixed signals and bleed-through problems across dyes. Existing spectral unmixing algorithms are not optimized for challenging biological specimens such as post-mortem human brain tissue, and often require manual intervention to extract spectral signatures. We therefore developed an intuitive, automated, and flexible package called SUFI: spectral unmixing of fluorescent images.

RESULTS

This package unmixes multispectral fluorescence images by automating the extraction of spectral signatures using vertex component analysis, and then performs one of three unmixing algorithms derived from remote sensing. We evaluate these remote sensing algorithms' performances on four unique biological datasets and compare the results to unmixing results obtained using ZEN Black software (Zeiss). We lastly integrate our unmixing pipeline into the computational tool dotdotdot, which is used to quantify individual RNA transcripts at single cell resolution in intact tissues and perform differential expression analysis, and thereby provide an end-to-end solution for multispectral fluorescence image analysis and quantification.

CONCLUSIONS

In summary, we provide a robust, automated pipeline to assist biologists with improved spectral unmixing of multispectral fluorescence images.

Design, fabrication, and preclinical testing of a miniaturized, multispectral, chip-on-tip, imaging probe for intraluminal fluorescence imaging of the gastrointestinal tract

Gastrointestinal cancers continue to account for a disproportionately large percentage of annual cancer deaths in the US. Advancements in miniature imaging technology combined with a need for precise and thorough tumor detection in gastrointestinal cancer screenings fuel the demand for new, small-scale, and low-cost methods of localization and margin identification with improved accuracy. Here, we report the development of a miniaturized, chip-on-tip, multispectral, fluorescence imaging probe designed to port through a gastroscope working channel with the aim of detecting cancerous lesions in point-of-care endoscopy of the gastrointestinal lumen. Preclinical testing has confirmed fluorescence sensitivity and supports that this miniature probe can locate structures of interest via detection of fluorescence emission from exogenous contrast agents. This work demonstrates the design and preliminary performance evaluation of a miniaturized, single-use, chip-on-tip fluorescence imaging system, capable of detecting multiple fluorochromes, and devised for deployment via the accessory channel of a standard gastroscope.

Multiplex Fluorescent Immunohistochemistry for Preservation of Tumor Microenvironment Architecture and Spatial Relationship of Cells in Tumor Tissues

The tumor microenvironment (TME), composed of immune cells, antigens, and local soluble factors, is integral to cancer development and progression. Traditional techniques such as immunohistochemistry, immunofluorescence, or flow cytometry limit the analysis of spatial data and cellular interactions within the TME, as they are restricted to colocalization of a small number of antigens or the loss of tissue architecture. Multiplex fluorescent immunohistochemistry (mfIHC) allows for detection of multiple antigens within a single tissue sample, providing a more comprehensive description of tissue composition and spatial interactions within the TME. This technique utilizes antigen retrieval, application of primary and secondary antibodies, followed by a tyramide-based chemical reaction to covalently bind a fluorophore to an epitope of interest and, eventually, stripping of the antibodies. This allows for multiple rounds of antibody application without concern for species cross-reactivity, as well as signal amplification which abrogates the autofluorescence that frequently plagues analysis of fixed tissues. As such, mfIHC can be used to quantify multiple cellular populations and their interactions, in situ, unlocking key biologic data that was previously unavailable. This chapter provides an overview of the experimental design, staining, and imaging strategies using a manual technique in formalin-fixed paraffin-embedded tissue sections.

Soybean cyst nematode detection and management: a review

Soybeans play a key role in global food security. U.S. soybean yields, which comprise [Formula: see text] of the total soybeans planted in the world, continue to experience unprecedented grain loss due to the soybean cyst nematode (SCN) plant pathogen. SCN remains one of the primary disruptive pests despite the existence of advanced management techniques such as crop rotation and SCN-resistant varieties. SCN detection is a key step in managing this disease; however, early detection is challenging because soybeans do not show any above ground symptoms unless they  are significantly damaged. Direct soil sampling remains the most common method for SCN detection, however, this method has several problems. For example, the threshold damage methods-adopted by most of the laboratories to make recommendations-is not reliable as it does not consider soil pH, N, P, and K values and relies solely on the egg count instead of assessment of the root infection. To overcome the challenges of manual soil sampling methods, deep learning and hyperspectral imaging are important current topics in precision agriculture for plant disease detection and have been proposed as cost-effective and efficient detection methods that can work at scale. We have reviewed more than 150 research papers focusing on soybean cyst nematodes with an emphasis on deep learning techniques for detection and management. First: we describe soybean vegetation and reproduction stages, SCN life cycles, and factors influencing this disease. Second: we highlight the impact of SCN on soybean yield loss and the challenges associated with its detection. Third: we describe direct sampling methods in which the soil samples are procured and analyzed to evaluate SCN egg counts. Fourth: we highlight the advantages and limitations of these direct methods, then review computer vision- and remote sensing-based detection methods: data collection using ground, aerial, and satellite approaches followed by a review of machine learning methods for image analysis-based soybean cyst nematode detection. We highlight the evaluation approaches and the advantages of overall detection workflow in high-performance and big data environments. Lastly, we discuss various management approaches, such as crop rotation, fertilization, SCN resistant varieties such as PI 88788, and SCN's increasing resistance to these strategies. We review machine learning approaches for soybean crop yield forecasting as well as the influence of pesticides, herbicides, and fertilizers on SCN infestation reduction. We provide recommendations for soybean research using deep learning and hyperspectral imaging to accommodate the lack of the ground truth data and training and testing methodologies, such as data augmentation and transfer learning, to achieve a high level of detection accuracy while keeping costs as low as possible.

The use of multispectral imaging for the discrimination of Arabica and Robusta coffee beans

Arabica coffee beans are sold at twice the price, or more, compared to Robusta beans and consequently are susceptible to economically motivated adulteration by substitution. There is a need for rapid, non-destructive, and efficient analytical techniques for monitoring the authenticity of Arabica coffee beans in the supply chain. In this study, multispectral imaging (MSI) was applied to discriminate roasted Arabica and Robusta coffee beans and perform quantitative prediction of Arabica coffee bean adulteration with Robusta. The Orthogonal Partial Least Squares Discriminant Analysis (OPLS-DA) model, built using selected spectral and morphological features from individual coffee beans, achieved 100% correct classification of the two coffee species in the test dataset. The OPLS regression model was able to successfully predict the level of adulteration of Arabica with Robusta. MSI analysis has potential as a rapid screening tool for the detection of fraud issues related to the authenticity of Arabica coffee beans.

Semantic segmentation of multispectral photoacoustic images using deep learning

Photoacoustic (PA) imaging has the potential to revolutionize functional medical imaging in healthcare due to the valuable information on tissue physiology contained in multispectral photoacoustic measurements. Clinical translation of the technology requires conversion of the high-dimensional acquired data into clinically relevant and interpretable information. In this work, we present a deep learning-based approach to semantic segmentation of multispectral photoacoustic images to facilitate image interpretability. Manually annotated photoacoustic and ultrasound imaging data are used as reference and enable the training of a deep learning-based segmentation algorithm in a supervised manner. Based on a validation study with experimentally acquired data from 16 healthy human volunteers, we show that automatic tissue segmentation can be used to create powerful analyses and visualizations of multispectral photoacoustic images. Due to the intuitive representation of high-dimensional information, such a preprocessing algorithm could be a valuable means to facilitate the clinical translation of photoacoustic imaging.

Spectroscopic photoacoustic microscopic imaging during single spatial scan using broadband excitation light pulses with wavelength-dependent time delay

In most multispectral optical-resolution photoacoustic microscopy (OR-PAM), spatial scanning is repeated for each excitation wavelength, which decreases throughput and causes motion artifacts during spectral processing. This study proposes a new spectroscopic OR-PAM technique to acquire information on the photoacoustic signal intensity and excitation wavelength from single spatial scans. The technique involves irradiating an imaging target with two broadband optical pulses with and without wavelength-dependent time delays. The excitation wavelength of the sample is then calculated by measuring the time delay between the photoacoustic signals generated by the two optical pulses. This technique is validated by measuring the excitation wavelengths of dyes in tubes. Furthermore, we demonstrate the three-dimensional spectroscopic OR-PAM of cells stained with suitable dyes. Although the tradeoff between excitation efficiency and excitation bandwidth must be adjusted based on the application, combining the proposed technique with fast spatial scanning methods can significantly contribute to recent OR-PAM applications, such as monitoring quick biological events and microscale tracking of moving materials.

MEDUSA: An open-source and webcam based multispectral imaging system

Multispectral imaging is at the forefront of contactless surface analysis. Standard multispectral imaging systems use sophisticated software, cameras and light filtering optics. This paper discloses the building of a customizable and cost-effective multispectral imaging and analysis system. It integrates a web camera, light emitting diodes (LEDs) lighting, a semisphere for even lightening, an open-source Arduino™ development board and a free Python application to automatically obtain and visually analyze multispectral images. The device is hereafter called MEDUSA and its optical performance was tested for repeated Imaging consistency, visible and near infrared band sensitivity and lighting evenness. Four proof of concept tests were run in order to understand the advantageous use of this system, as compared to a simple visual score of diverse samples. Each of three qualitative tests used sets of 12 LED band spectral images to analyze ink changes in a counterfeit bill, surface bruises on Hass avocado fruits and transient changes in petri dish grown bacterial colonies. A fourth test used single band imaging in a set of standard laboratory analyzed plant samples, to quantitatively relate a red band light reflectance to its nitrogen content. These tests indicate that MEDUSA made images may yield qualitative and quantitative spectral information unseen to the naked eye, suggesting potential use in currency counterfeit tests, food quality analyses, microbial phenotyping and agricultural plant chemistry. MEDUSA can be freely reproduced and customized from this research, making it a powerful and affordable analytical tool to analyze a wide range of subtle chemical properties in samples at industrial and science fields.

Machine learning analysis of multispectral imaging and clinical risk factors to predict amputation wound healing

OBJECTIVE

Prediction of amputation wound healing is challenging due to the multifactorial nature of critical limb ischemia and lack of objective assessment tools. Up to one-third of amputations require revision to a more proximal level within 1 year. We tested a novel wound imaging system to predict amputation wound healing at initial evaluation.

METHODS

Patients planned to undergo amputation due to critical limb ischemia were prospectively enrolled. Clinicians evaluated the patients in traditional fashion, and all clinical decisions for amputation level were determined by the clinician's judgement. Multispectral images of the lower extremity were obtained preoperatively using a novel wound imaging system. Clinicians were blinded to the machine analysis. A standardized wound healing assessment was performed on postoperative day 30 by physical exam to determine whether the amputation site achieved complete healing. If operative revision or higher level of amputation was required, this was undertaken based solely upon the provider's clinical judgement. A machine learning algorithm combining the multispectral imaging data with patient clinical risk factors was trained and tested using cross-validation to measure the wound imaging system's accuracy of predicting amputation wound healing.

RESULTS

A total of 22 patients undergoing 25 amputations (10 toe, five transmetatarsal, eight below-knee, and two above-knee amputations) were enrolled. Eleven amputations (44%) were non-healing after 30 days. The machine learning algorithm had 91% sensitivity and 86% specificity for prediction of non-healing amputation sites (area under curve, 0.89).

CONCLUSIONS

This pilot study suggests that a machine learning algorithm combining multispectral wound imaging with patient clinical risk factors may improve prediction of amputation wound healing and therefore decrease the need for reoperation and incidence of delayed healing. We propose that this, in turn, may offer significant cost savings to the patient and health system in addition to decreasing length of stay for patients.

Review on Optical Imaging Techniques for Multispectral Analysis of Nanomaterials

Biomedical imaging is an essential tool for investigating biological responses in vivo. Among the several imaging techniques, optical imaging systems with multispectral analysis of nanoparticles have been widely investigated due to their ability to distinguish the substances in biological tissues in vivo. This review article focus on multispectral optical imaging techniques that can provide molecular functional information. We summarize the basic principle of the spectral unmixing technique that enables the delineation of optical chromophores. Then, we explore the principle, typical system configuration, and biomedical applications of the representative optical imaging techniques, which are fluorescence imaging, two-photon microscopy, and photoacoustic imaging. The results in the recent studies show the great potential of the multispectral analysis techniques for monitoring responses of biological systems in vivo.

Identification of a bio-signature for barley resistance against Pyrenophora teres infection based on physiological, molecular and sensor-based phenotyping

Necrotic and chlorotic symptoms induced during Pyrenophora teres infection in barley leaves indicate a compatible interaction that allows the hemi-biotrophic fungus Pyrenophora teres to colonise the host. However, it is unexplored how this fungus affects the physiological responses of resistant and susceptible cultivars during infection. To assess the degree of resistance in four different cultivars, we quantified visible symptoms and fungal DNA and performed expression analyses of genes involved in plant defence and ROS scavenging. To obtain insight into the interaction between fungus and host, we determined the activity of 19 key enzymes of carbohydrate and antioxidant metabolism. The pathogen impact was also phenotyped non-invasively by sensor-based multireflectance and -fluorescence imaging. Symptoms, regulation of stress-related genes and pathogen DNA content distinguished the cultivar Guld as being resistant. Severity of net blotch symptoms was also strongly correlated with the dynamics of enzyme activities already within the first day of infection. In contrast to the resistant cultivar, the three susceptible cultivars showed a higher reflectance over seven spectral bands and higher fluorescence intensities at specific excitation wavelengths. The combination of semi high-throughput physiological and molecular analyses with non-invasive phenotyping enabled the identification of bio-signatures that discriminates the resistant from susceptible cultivars.

Water and nitrogen in-situ imaging detection in live corn leaves using near-infrared camera and interference filter

BACKGROUND

Realizing imaging detection of water and nitrogen content in different regions of plant leaves in-site and real-time can provide an efficient new technology for determining crop drought resistance and nutrient regulation mechanisms, or for use in precision agriculture. Near-infrared imaging is the preferred technology for in-situ real-time detection owing to its non-destructive nature; moreover, it provides rich information. However, the use of hyperspectral imaging technology is limited as it is difficult to use it in field because of its high weight and power.

RESULTS

We developed a smart imaging device using a near-infrared camera and an interference filter; it has a low weight, requires low power, and has a multi-wavelength resolution. The characteristic wavelengths of the filter that realize leaf moisture measurement are 1150 and 1400 nm, respectively, the characteristic wavelength of the filter that realizes nitrogen measurement is 1500 nm, and all filter bandwidths are 25 nm. The prediction result of the average leaf water content model obtained with the device was R² = 0.930, RMSE = 1.030%; the prediction result of the average nitrogen content model was R² = 0.750, RMSE = 0.263 g.

CONCLUSIONS

Using the average water and nitrogen content model, an image of distribution of water and nitrogen in different areas of corn leaf was obtained, and its distribution characteristics were consistent with the actual leaf conditions. The experimental materials used in this research were fresh leaves in the field, and the test was completed indoors. Further verification of applying the device and model to the field is underway.

Optimized tumour infiltrating lymphocyte assessment for triple negative breast cancer prognostics

The tumour microenvironment has been shown to be a valuable source of prognostic information for different cancer types. This holds in particular for triple negative breast cancer (TNBC), a breast cancer subtype for which currently no prognostic biomarkers are established. Although different methods to assess tumour infiltrating lymphocytes (TILs) have been published, it remains unclear which method (marker, region) yields the most optimal prognostic information. In addition, to date, no objective TILs assessment methods are available. For this proof of concept study, a subset of our previously described TNBC cohort (n = 94) was stained for CD3, CD8 and FOXP3 using multiplex immunohistochemistry and subsequently imaged by a multispectral imaging system. Advanced whole-slide image analysis algorithms, including convolutional neural networks (CNN) were used to register unmixed multispectral images and corresponding H&E sections, to segment the different tissue compartments (tumour, stroma) and to detect all individual positive lymphocytes. Densities of positive lymphocytes were analysed in different regions within the tumour and its neighbouring environment and correlated to relapse free survival (RFS) and overall survival (OS). We found that for all TILs markers the presence of a high density of positive cells correlated with an improved survival. None of the TILs markers was superior to the others. The results of TILs assessment in the various regions did not show marked differences between each other. The negative correlation between TILs and survival in our cohort are in line with previous studies. Our results provide directions for optimizing TILs assessment methodology.

Spoilage assessment of chicken breast fillets by means of fourier transform infrared spectroscopy and multispectral image analysis

The objective of this research was the evaluation of Fourier transforms infrared spectroscopy (FT-IR) and multispectral image analysis (MSI) as efficient spectroscopic methods in tandem with multivariate data analysis and machine learning for the assessment of spoilage on the surface of chicken breast fillets. For this purpose, two independent storage experiments of chicken breast fillets (n ​= ​215) were conducted at 0, 5, 10, and 15 ​°C for up to 480 ​h. During storage, samples were analyzed microbiologically for the enumeration of Total Viable Counts (TVC) and Pseudomonas spp. In addition, FT-IR and MSI spectral data were collected at the same time intervals as for microbiological analyses. Multivariate data analysis was performed using two software platforms (a commercial and a publicly available developed platform) comprising several machine learning algorithms for the estimation of the TVC and Pseudomonas spp. population of the surface of the samples. The performance of the developed models was evaluated by intra batch and independent batch testing. Partial Least Squares- Regression (PLS-R) models from the commercial software predicted TVC with root mean square error (RMSE) values of 1.359 and 1.029 log CFU/cm² for MSI and FT-IR analysis, respectively. Moreover, RMSE values for Pseudomonas spp. model were 1.574 log CFU/cm² for MSI data and 1.078 log CFU/cm² for FT-IR data. From the implementation of the in-house sorfML platform, artificial neural networks (nnet) and least-angle regression (lars) were the most accurate models with the best performance in terms of RMSE values. Nnet models developed on MSI data demonstrated the lowest RMSE values (0.717 log CFU/cm²) for intra-batch testing, while lars outperformed nnet on independent batch testing with RMSE of 1.252 log CFU/cm². Furthermore, lars models excelled with the FT-IR data with RMSE of 0.904 and 0.851 log CFU/cm² in intra-batch and independent batch testing, respectively. These findings suggested that FT-IR analysis is more efficient than MSI to predict the microbiological quality on the surface of chicken breast fillets.

•

Poultry meat’s vulnerability to spoilage demands rapid quality assessment LWT-Food Sci. Technol.methods.

•

FT-IR and MSI are non-invasive methods applied in a variety of meat products.

•

SorfML is a web platform providing diverse machine learning algorithms.

•

FT-IR analysis via lars predicted efficiently microbial loads of TVC.

Ultraviolet vision aids the detection of nutrient-dense non-signaling plant foods

To expand our understanding of what tasks are particularly helped by UV vision and may justify the costs of focusing high-energy light onto the retina, we used an avian-vision multispectral camera to image diverse vegetated habitats in search of UV contrasts that differ markedly from visible-light contrasts. One UV contrast that stood out as very different from visible-light contrasts was that of nutrient-dense non-signaling plant foods (such as young leaves and immature fruits) against their natural backgrounds. From our images, we calculated color contrasts between 62+ species of such foods and mature foliage for the two predominant color vision systems of birds, UVS and VS. We also computationally generated images of what a generalized tetrachromat, unfiltered by oil droplets, would see, by developing a new methodology that uses constrained linear least squares to solve for optimal weighted combinations of avian camera filters to mimic new spectral sensitivities. In all visual systems, we found that nutrient-dense non-signaling plant foods presented a lower, often negative figure-ground contrast in the UV channels, and a higher, often positive figure-ground contrast in the visible channels. Although a zero contrast may sound unhelpful, it can actually enhance color contrast when compared in a color opponent system to other channels with nonzero contrasts. Here, low or negative UV contrasts markedly enhanced color contrasts. We propose that plants may struggle to evolve better UV crypsis since UV reflectance from vegetation is largely specular and thus highly dependent on object orientation, shape, and texture.

Deep multispectral image registration network

Multispectral imaging (MSI) of the ocular fundus provides a sequence of narrow-band images to show the different depths in the retina and choroid. One challenge in analyzing MSI images comes from the image-to-image spatial misalignment, which occurs because the acquisition time of eye MSI images is commonly longer than the natural time scale of the eye's saccadic movement. It is necessary to align images because ophthalmologists usually overlay two of the images to analyze specific features when analyzing MSI images. In this paper, we propose a weakly supervised MSI image registration network, called MSI-R-NET, for multispectral fundus image registration. Compared to other deep-learning-based registration methods, MSI-R-NET utilizes the blood vessel segmentation label to provide spatial correspondence. In addition, we employ a feature equilibrium module to connect the aggregating layers better, and propose a multiresolution auto-context structure to adapt the registration task. In the testing stage, given a new pair of MSI images, the trained model can predict the pixelwise spatial correspondence without labeled blood vessel information. The experimental results demonstrate that the proposed segmentation-driven registration method is highly accurate.

Characteristics of punctate inner choroidopathy complicated by choroidal neovascularisation on Multispectral Imaging in comparison with other imaging modalities

PURPOSE

To describe the characteristics seen on multispectral imaging (MSI) in patients with punctate inner choroidopathy (PIC) and choroidal neovascularisation (CNV) and compare the findings with current standard multimodal imaging techniques.

METHODS

This is a retrospective observational case series of 10 patients with PIC complicated by CNV that underwent multimodal retinal imaging examinations.

RESULTS

Twelve eyes of 10 patients were included. CNV was identified in 11 of the 12 eyes (91.7%) by MSI with nodular or trunk-like hyperreflectance on retinal oxy/deoxyhemoglobin map. MSI revealed choroidal vasculature around CNV in 91.7% eyes and pathological changes including retinal pigment epithelial atrophy and melanin disruption of punctate lesions in all eyes.

CONCLUSION

MSI helps in noninvasively detecting CNV in PIC patients and observing associated changes in choroidal vasculature. This imaging technique is also a promising tool for better tracking pathological changes of PIC lesions complementary to current standard multimodal imaging modalities.

Multispectral imaging technology: Visualize, analyze, phenotyping, and quantify immune cells in situ

In the personalized medicine era, the field of immunohistopathology is evolving to provide even more precise diagnostic information to efficiently apply targeting therapies. In this regard, MultiSpectral fluorescence Imaging (MSI) is a powerful and reliable technique that provides a detailed and remarkable analysis of multiple biomarkers within their histological context. In particular, the analysis of the immune infiltrate in conjunction with the expression of immune checkpoint molecules could explain why the efficacy of the promising treatments based on immune modulator monoclonal antibodies is still limited. We analyzed the advantages and the pitfalls of applying MSI technology to investigate the immune infiltrate in correlation with programmed death-ligand 1 expression in paraffin embedded ovarian cancer samples.

On the application of multi-parametric optical phenotyping of bacterial colonies for multipurpose microbiological diagnostics

The development of new diagnostics techniques and modalities is critical for early detection of microbial contamination. In this study, the novel integrated system for multi-parametric optical phenotyping and characterization of bacterial colonies, is presented. The system combines Mach-Zehnder interferometer with a spectral imaging system for capturing multispectral diffraction patterns and multispectral two-dimensional transmission maps of bacterial colonies, along with the simultaneous interferometric profilometry. The herein presented investigation was carried out on five representative bacteria species and nearly 3000 registered multispectral optical signatures. The interferograms were analyzed by four-step phase shift algorithm to reconstruct the colony profile to enable the obtaining of the comparable optical signatures. The dedicated image processing algorithms were used for extraction of quantitative features of these signatures. The random forest algorithm was applied for selection of the most predictive set of features, which were used in classification model based on Support-Vector Machine. Obtained results have shown that the use of multiple multispectral optical signatures provide a multi-parametric bacteria identification at an exceptionally high accuracy (99.4-100%), significantly better than in case of classification based on each of these signatures (multispectral diffraction patterns, two-dimensional transmission coefficient maps), separately. Obtained results revealed that analysis of multispectral signatures can also be applied for characterisation of physical, physicochemical and chemical properties of the bacterial colonies in the presence of the antimicrobial factors. Therefore, the proposed label-free, non-destructive optical technique has perspectives to be exploited in the multipurpose diagnostics and it can be used as a pre-screening tool in microbiological laboratories.

Novel observations in choroidal osteoma by multispectral imaging: a pilot case series

PURPOSE

To identify novel tumor-specific features of ossification by using multispectral imaging (MSI) in patients diagnosed with choroidal osteoma.

METHODS

Six eyes of 5 patients previously diagnosed with choroidal osteoma by ocular ultrasonography and orbital computerized tomography were observed with multispectral imaging (MSI). Traditional multimodal imaging, including color fundus photograph (CFP) and enhanced depth-imaging-optical coherence tomography (EDI-OCT), fundus autofluorescence (FAF), indocyanine green angiography/fundus fluorescein angiography (ICGA/FFA), was performed. Osseous features detected by MSI such as calcification and decalcification were characterized and compared with other imaging modalities.

RESULTS

In all 3 eyes with calcified choroidal osteoma (100%), MSI featured by the homogeneous reflectance in 550 nm but the beehive appearance in 600-680 nm and homogenous hyper-reflectance in 780-850 nm', indicating the compact bone in the outer layers and bone trabecula in the middle layer (Sandwich sign). The pigmentary change showed high agreement between MSI and FAF. In other 3 eyes with extensive decalcification, MSI was able to differentiate the inactive portion of the osteoma from the decalcified area. The inactive portion was characterized by geographic hyper-reflective islands with higher reflectivity border (floating island sign). Decalcified portion was featured by increased definition and reflectivity from osteoma. Partial decalcification and total decalcification can be differentiated in one decalcifying eye (33.3%). MSI revealed better the presence and border of the osteoma compared with FFA, FAF and MC (100%) in all six eyes in our study.

CONCLUSIONS

MSI presented characteristic osseous-related features of choroidal osteoma, providing clear evidence for differentiating osteoblastic and osteoclastic regions and noncalcifying regions. It can contribute to en-face visualization of choroidal osteomas at different stages, providing new insight into the spectrum behavior of CO.

Surgical spectral imaging

Recent technological developments have resulted in the availability of miniaturised spectral imaging sensors capable of operating in the multi- (MSI) and hyperspectral imaging (HSI) regimes. Simultaneous advances in image-processing techniques and artificial intelligence (AI), especially in machine learning and deep learning, have made these data-rich modalities highly attractive as a means of extracting biological information non-destructively. Surgery in particular is poised to benefit from this, as spectrally-resolved tissue optical properties can offer enhanced contrast as well as diagnostic and guidance information during interventions. This is particularly relevant for procedures where inherent contrast is low under standard white light visualisation. This review summarises recent work in surgical spectral imaging (SSI) techniques, taken from Pubmed, Google Scholar and arXiv searches spanning the period 2013-2019. New hardware, optimised for use in both open and minimally-invasive surgery (MIS), is described, and recent commercial activity is summarised. Computational approaches to extract spectral information from conventional colour images are reviewed, as tip-mounted cameras become more commonplace in MIS. Model-based and machine learning methods of data analysis are discussed in addition to simulation, phantom and clinical validation experiments. A wide variety of surgical pilot studies are reported but it is apparent that further work is needed to quantify the clinical value of MSI/HSI. The current trend toward data-driven analysis emphasises the importance of widely-available, standardised spectral imaging datasets, which will aid understanding of variability across organs and patients, and drive clinical translation.

Light source calibration for multispectral imaging in surgery

PURPOSE

Live intra-operative functional imaging has multiple potential clinical applications, such as localization of ischemia, assessment of organ transplantation success and perfusion monitoring. Recent research has shown that live monitoring of functional tissue properties, such as tissue oxygenation and blood volume fraction, is possible using multispectral imaging in laparoscopic surgery. While the illuminant spectrum is typically kept constant in laparoscopic surgery and can thus be estimated from preoperative calibration images, a key challenge in open surgery originates from the dynamic changes of lighting conditions.

METHODS

The present paper addresses this challenge with a novel approach to light source calibration based on specular highlight analysis. It involves the acquisition of low-exposure time images serving as a basis for recovering the illuminant spectrum from pixels that contain a dominant specular reflectance component.

RESULTS

Comprehensive in silico and in vivo experiments with a range of different light sources demonstrate that our approach enables an accurate and robust recovery of the illuminant spectrum in the field of view of the camera, which results in reduced errors with respect to the estimation of functional tissue properties. Our approach further outperforms state-of-the-art methods proposed in the field of computer vision.

CONCLUSION

Our results suggest that low-exposure multispectral images are well suited for light source calibration via specular highlight analysis. This work thus provides an important first step toward live functional imaging in open surgery.

PCA-based method for managing and analyzing single-spot analysis referenced to spectral imaging for artworks diagnostics

Artworks diagnostics is based on the joint use of several nondestructive techniques to acquire complementary information on the materials. A common practice in the field is to perform the analyses with single-spot analytical techniques, e.g. spectroscopy-based, after a preliminary screening of the artwork with full-field imaging-based techniques. We present a method and its practical implementation for fusing and analyzing data collected using analytical systems that acquire single spot measurements mapped to spectral imaging stacks. The fused dataset of single-spot and imaging observations is analyzed using principal component analysis (PCA). The effectiveness of the method for artworks diagnostics is shown on spectroscopy and imaging datasets of an ancient canvas painting. The results of the PCA analysis on the final fused dataset are compared against the PCA analysis performed on the original datasets from single-spot and imaging measurements taken separately. We propose two practical implementations of the procedure, one based on using graphical user interface (GUI) and open-source GIS software (QGIS), the other one based on an open-source Python module, named SPOLVERRO, specifically developed for this project and released on a public repository. The method allows conservation scientists to analize effectively the heterogeneous datasets acquired in a diagnostic campaign.

•

single-spot spectroscopy data are referenced on imaging data.

•

the sampling area of each spectroscopy spot is used for extracting and averaging the respective imaging data values.

•

the final matrix is analyzed using PCA for extracting further information.

The vascular occlusion test using multispectral imaging: a validation study : The VASOIMAGE study

Multispectral imaging (MSI) is a new, non-invasive method to continuously measure oxygenation and microcirculatory perfusion, but has limitedly been validated in healthy volunteers. The present study aimed to validate the potential of multispectral imaging in the detection of microcirculatory perfusion disturbances during a vascular occlusion test (VOT). Two consecutive VOT's were performed on healthy volunteers and tissue oxygenation was measured with MSI and near-infrared spectroscopy (NIRS). Correlations between the rate of desaturation, recovery and the hyperemic area under the curve (AUC) measured by MSI and NIRS were calculated. Fifty-eight volunteers were included. The MSI oxygenation curves showed identifiable components of the VOT, including a desaturation and recovery slope and hyperemic area under the curve, similar to those measured with NIRS. The correlation between the rate of desaturation measured by MSI and NIRS was moderate: r = 0.42 (p = 0.001) for the first and r = 0.41 (p = 0.002) for the second test. Our results suggest that non-contact multispectral imaging is able to measure changes in regional oxygenation and deoxygenation during a vascular occlusion test in healthy volunteers. When compared to measurements with NIRS, correlation of results was moderate to weak, most likely reflecting differences in physiology of the regions of interest and measurement technique.

Non-destructive identification of single hard seed via multispectral imaging analysis in six legume species

BACKGROUND

Physical dormancy (hard seed) occurs in most species of Leguminosae family and has great consequences not only for ecological adaptation but also for agricultural practice of these species. A rapid, nondestructive and on-site screening method to detect hard seed within species is fundamental important for maintaining seed vigor and germplasm storage as well as understanding seed adaptation to various environment. In this study, the potential of multispectral imaging with object-wise multivariate image analysis was evaluated as a way to identify hard and soft seeds in Acacia seyal, Galega orientulis, Glycyrrhiza glabra, Medicago sativa, Melilotus officinalis, and Thermopsis lanceolata. Principal component analysis (PCA), linear discrimination analysis (LDA), and support vector machines (SVM) methods were applied to classify hard and soft seeds according to their morphological features and spectral traits.

RESULTS

The performance of discrimination model via multispectral imaging analysis was varied with species. For M. officinalis, M. sativa, and G. orientulis, an excellent classification could be achieved in an independent validation data set. LDA model had the best calibration and validation abilities with the accuracy up to 90% for M. sativa. SVM got excellent seed discrimination results with classification accuracy of 91.67% and 87.5% for M. officinalis and G. orientulis, respectively. However, both LDA and SVM model failed to discriminate hard and soft seeds in A. seyal, G. glabra, and T. lanceolate.

CONCLUSIONS

Multispectral imaging together with multivariate analysis could be a promising technique to identify single hard seed in some legume species with high efficiency. More legume species with physical dormancy need to be studied in future research to extend the use of multispectral imaging techniques.

[The development of real-time multispectral imaging for the diagnostics of bladder cancer]

The performance of white light (WL) cystoscopy in the diagnostics of bladder cancer can be optimized by the use of modern imaging modalities, such as photodynamic diagnostics (PDD) and narrow band imaging (NBI). Real-time multispectral imaging (rMSI) enables simultaneous imaging of reflectance and fluorescence modalities in multiple spectral bands. We created a multiparametric cystoscopy image by digital overlapping of several modalities, e.g. WL, enhanced vascular contrast (EVC), raw fluorescence mode, protoporphyrin IX and autofluorescence (AF). The technical development and the subsequent clinical implementation of rMSI required a structured preclinical evaluation process, including both ex vivo and in vivo trials before the technology can be applied in patients. This review article presents the phases of testing, validation and the first clinical application of rMSI in urological endoscopy.

Multiplex immunohistochemistry as a novel tool for the topographic assessment of the bone marrow stem cell niche

Immunohistochemistry (IHC) using specific antibodies is a well-established method for the visualization of distinct cell populations. With increasing availability of suitable methods for complex tissue analyses, new demands have arisen to provide next to complex quantitative data information on protein expression, spatial distribution and cell-cell interactions in tissue sections. During the last decade, tissue preparation, fluorescent dyes, hardware imaging and software analysis were improved to solve problems concerning quantitative preciseness and tissue autofluorescence of multicolor staining. Automated cell segmentation as well as subcellular multiparameter analysis of fluorescence-based multiplexed IHC techniques, such as multispectral imaging (MSI), allows the quantification and localization of multiple proteins in the same tissue section. This technique gives us the opportunity to visualize and record the spatial relationship between different cells and is currently employed for formalin-fixed, paraffin-embedded (FFPE) samples, but has not yet been developed for calcified bone marrow (BM) biopsies. This chapter summarizes a novel protocol developed for decalcified FFPE BM samples. In addition, it discusses the technical aspects and pitfalls using this material thereby extending the use of MSI for analysis of BM malignancies. It provides an overview on the characterization and distribution of cell populations and protein expression patterns regarding their prognostic and predictive value, and their use for guidance of therapeutic decisions.

Phenolic distribution in apple epidermal and outer cortex tissue by multispectral deep-UV autofluorescence cryo-imaging

Phenolic compounds in fruit are involved in responses to biotic and abiotic stresses and are responsible for organoleptic properties. To establish the distribution of these secondary metabolites at the tissue and sub-cellular scales, mapping of fluorescence in apple epidermis and outer cortex tissue in cryogenic condition was performed after deep-UV excitation at 275 nm. Douce Moën and Guillevic cider apple varieties were sampled and frozen after harvest, after 30 days at 4 °C and after 20 days at room temperature. Image analysis of fluorescence emission images acquired between 300 and 650 nm allowed the assignment of fluorescence signals to phenolic compound families based on reference molecules. Emission attributed to monomeric and/or condensed flavanol was localized in whole tissue with major fluorescence in the cuticle region. Hydroxycinnamic acids were found predominantly in the outer cortex and appeared in the cell wall. Fluorescent pigments were mostly found in the epidermis. The distribution of flavanols in the sub-cuticle and phenolic acids in the outer cortex distinguished apple varieties. Storage conditions had no impact on phenolic distribution. The proposed fluorescent imaging and analysis approach enables studies on phenolic distribution in relation to fruit development, biotic/abiotic stress resistance and quality.

Analytical and mathematical methods for revealing hidden details in ancient manuscripts and paintings: A review

In this work, a critical review of the current nondestructive probing and image analysis approaches is presented, to revealing otherwise invisible or hardly discernible details in manuscripts and paintings relevant to cultural heritage and archaeology. Multispectral imaging, X-ray fluorescence, Laser-Induced Breakdown Spectroscopy, Raman spectroscopy and Thermography are considered, as techniques for acquiring images and spectral image sets; statistical methods for the analysis of these images are then discussed, including blind separation and false colour techniques. Several case studies are presented, with particular attention dedicated to the approaches that appear most promising for future applications. Some of the techniques described herein are likely to replace, in the near future, classical digital photography in the study of ancient manuscripts and paintings.

Uncertainty-aware performance assessment of optical imaging modalities with invertible neural networks

PURPOSE

Optical imaging is evolving as a key technique for advanced sensing in the operating room. Recent research has shown that machine learning algorithms can be used to address the inverse problem of converting pixel-wise multispectral reflectance measurements to underlying tissue parameters, such as oxygenation. Assessment of the specific hardware used in conjunction with such algorithms, however, has not properly addressed the possibility that the problem may be ill-posed.

METHODS

We present a novel approach to the assessment of optical imaging modalities, which is sensitive to the different types of uncertainties that may occur when inferring tissue parameters. Based on the concept of invertible neural networks, our framework goes beyond point estimates and maps each multispectral measurement to a full posterior probability distribution which is capable of representing ambiguity in the solution via multiple modes. Performance metrics for a hardware setup can then be computed from the characteristics of the posteriors.

RESULTS

Application of the assessment framework to the specific use case of camera selection for physiological parameter estimation yields the following insights: (1) estimation of tissue oxygenation from multispectral images is a well-posed problem, while (2) blood volume fraction may not be recovered without ambiguity. (3) In general, ambiguity may be reduced by increasing the number of spectral bands in the camera.

CONCLUSION

Our method could help to optimize optical camera design in an application-specific manner.

Utilization of computer vision and multispectral imaging techniques for classification of cowpea (Vigna unguiculata) seeds

BACKGROUND

The traditional methods for evaluating seeds are usually performed through destructive sampling followed by physical, physiological, biochemical and molecular determinations. Whilst proven to be effective, these approaches can be criticized as being destructive, time consuming, labor intensive and requiring experienced seed analysts. Thus, the objective of this study was to investigate the potential of computer vision and multispectral imaging systems supported with multivariate analysis for high-throughput classification of cowpea (Vigna unguiculata) seeds. An automated computer-vision germination system was utilized for uninterrupted monitoring of seeds during imbibition and germination to identify different categories of all individual seeds. By using spectral signatures of single cowpea seeds extracted from multispectral images, different multivariate analysis models based on linear discriminant analysis (LDA) were developed for classifying the seeds into different categories according to ageing, viability, seedling condition and speed of germination.

RESULTS

The results revealed that the LDA models had good accuracy in distinguishing 'Aged' and 'Non-aged' seeds with an overall correct classification (OCC) of 97.51, 96.76 and 97%, 'Germinated' and 'Non-germinated' seeds with OCC of 81.80, 79.05 and 81.0%, 'Early germinated', 'Medium germinated' and 'Dead' seeds with OCC of 77.21, 74.93 and 68.00% and among seeds that give 'Normal' and 'Abnormal' seedlings with OCC of 68.08, 64.34 and 62.00% in training, cross-validation and independent validation data sets, respectively. Image processing routines were also developed to exploit the full power of the multispectral imaging system in visualizing the difference among seed categories by applying the discriminant model in a pixel-wise manner.

CONCLUSION

The results demonstrated the capability of the multispectral imaging system in the ultraviolet, visible and shortwave near infrared range to provide the required information necessary for the discrimination of individual cowpea seeds to different classes. Considering the short time of image acquisition and limited sample preparation, this stat-of-the art multispectral imaging method and chemometric analysis in classifying seeds could be a valuable tool for on-line classification protocols in cost-effective real-time sorting and grading processes as it provides not only morphological and physical features but also chemical information for the seeds being examined. Implementing image processing algorithms specific for seed quality assessment along with the declining cost and increasing power of computer hardware is very efficient to make the development of such computer-integrated systems more attractive in automatic inspection of seed quality.

Multispectral Fluorescence Imaging Allows for Distinctive Topographic Assessment and Subclassification of Tumor-Infiltrating and Surrounding Immune Cells

Histomorphology has significantly changed over the last decades due to technological achievements in immunohistochemistry (IHC) for the visualization of specific proteins and in molecular pathology, particularly in the field of in situ hybridization of small oligonucleotides and amplification of DNA and RNA amplicons. With an increased availability of suitable methods, the demands regarding the observer of histomorphological slides were the supply of complex quantitative data as well as more information about protein expression and cell-cell interactions in tissue sections. Advances in fluorescence-based multiplexed IHC techniques, such as multispectral imaging (MSI), allow the quantification of multiple proteins at the same tissue section. In histopathology, it is a well-known technique for over a decade yet harboring serious problems concerning quantitative preciseness and tissue autofluorescence of multicolor staining when using formalin-fixed, paraffin-embedded (FFPE) tissue specimen. In recent years, milestones in tissue preparation, fluorescent dyes, hardware imaging, and software analysis were achieved including automated tissue segmentation (e.g., tumor vs. stroma) as well as in cellular and subcellular multiparameter analysis.This chapter covers the role that MSI plays in anatomic pathology for the analysis of FFPE tissue sections, discusses the technical aspects of MSI, and provides a review of its application in the characterization of immune cell infiltrates and beyond regarding its prognostic and predictive value and its use for guidance of clinical decisions for immunotherapeutic strategies.

Microfluidic Free-Flow Isoelectric Focusing with Real-Time pI Determination

Free-flow electrophoresis (FFE) may be used for continuous and preparative separation of a wide variety of biomolecules. Isoelectric focusing (IEF) provides for the separation of compounds according to their isoelectric point (pI). Here we describe a microfluidic chip-based protocol for the fabrication, application, and optical monitoring of free-flow isoelectric focusing (FFIEF) of proteins and peptides on the microscale with optical surveillance of the microscopic pH gradient provided by an integrated pH sensing layer. This protocol may be used with modifications also for the FFIEF of other biomolecules and may serve as template for the fabrication of microfluidic chips with integrated fluorescent or luminescent pH sensor layers for FFE and other applications.

Evaluation of Fourier transform infrared spectroscopy and multispectral imaging as means of estimating the microbiological spoilage of farmed sea bream

The objective of the present study was the evaluation of Fourier transform infrared (FTIR) spectroscopy and multispectral imaging (MSI), in tandem with multivariate data analysis, as means of estimating the microbiological quality of sea bream. Farmed whole ungutted fish were stored aerobically at 0, 4 and 8 °C. At regular time intervals, fish samples (i.e. cut portions) were analysed microbiologically, while FTIR and MSI measurements also were acquired at both the skin and flesh sides of the samples. Partial least squares regression (PLSR) models were calibrated to provide quantitative estimations of the microbiological status of fish based on spectral data, in a temperature-independent manner. The PLSR model based on the FTIR data of fish skin exhibited good performance when externally validated, with the coefficient of determination (R²) and the root mean square error (RMSE) being 0.727 and 0.717, respectively. Hence, FTIR spectroscopy appears to be promising for the rapid and non-invasive monitoring of the microbiological spoilage of whole sea bream. Contrarily, the MSI models' performance was unsatisfactory, delimitating their potential exploitation in whole fish quality assessment. Model optimization results concerning fish flesh indicated that MSI may be propitious in skinned fish products, with its definite competence warranting further investigation.

A method for real-time classification of insect vectors of mosaic and brown streak disease in cassava plants for future implementation within a low-cost, handheld, in-field multispectral imaging sensor

BACKGROUND

The paper introduces a multispectral imaging system and data-processing approach for the identification and discrimination of morphologically indistinguishable cryptic species of the destructive crop pest, the whitefly Bemisia tabaci. This investigation and the corresponding system design, was undertaken in two phases under controlled laboratory conditions. The first exploited a prototype benchtop variant of the proposed sensor system to analyse four cryptic species of whitefly reared under similar conditions. The second phase, of the methodology development, employed a commercial high-precision laboratory hyperspectral imager to recover reference data from five cryptic species of whitefly, immobilized through flash freezing, and taken from across four feeding environments.

RESULTS

The initial results, for the single feeding environment, showed that a correct species classification could be achieved in 85-95% of cases, utilising linear Partial Least Squares approaches. The robustness of the classification approach was then extended both in terms of the automated spatial extraction of the most pertinent insect body parts, to assist with the spectral classification model, as well as the incorporation of a non-linear Support Vector Classifier to maintain the overall classification accuracy at 88-98%, irrespective of the feeding and crop environment.

CONCLUSION

This study demonstrates that through an integration of both the spatial data, associated with the multispectral images being used to separate different regions of the insect, and subsequent spectral analysis of those sub-regions, that B. tabaci viral vectors can be differentiated from other cryptic species, that appear morphologically indistinguishable to a human observer, with an accuracy of up to 98%. The implications for the engineering design for an in-field, handheld, sensor system is discussed with respect to the learning gained from this initial stage of the methodology development.