Self-Organizing Map and Relational Perspective Mapping for the Accurate Visualization of High-Dimensional Hyperspectral Data

Coupling hyperspectral imaging with machine learning algorithms for detecting polyethylene (PE) and polyamide (PA) in soils

Soil, particularly in agricultural regions, has been recognized as one of the significant reservoirs for the emerging contaminant of MPs. Therefore, developing a rapid and efficient method is critical for their identification in soil. Here, we coupled HSI systems [i.e., VNIR (400-1000 nm), InGaAs (800-1600 nm), and MCT (1000-2500 nm)] with machine learning algorithms to distinguish soils spiked with white PE and PA (average size of 50 and 300 µm, respectively). The soil-normalized SWIR spectra unveiled significant spectral differences not only between control soil and pure MPs (i.e., PE 100% and PA 100%) but also among five soil-MPs mixtures (i.e., PE 1.6%, PE 6.9%, PA 5.0%, and PA 11.3%). This was primarily attributable to the 1st-3rd overtones and combination bands of C-H groups in MPs. Feature reductions visually demonstrated the separability of seven sample types by SWIR and the inseparability of five soil-MPs mixtures by VNIR. The detection models achieved higher accuracies using InGaAs (92-100%) and MCT (97-100%) compared to VNIR (44-87%), classifying 7 sample types. Our study indicated the feasibility of InGaAs and MCT HSI systems in detecting PE (as low as 1.6%) and PA (as low as 5.0%) in soil. SYNOPSIS: One of two SWIR HSI systems (i.e., InGaAs and MCT) with a sample imaging surface area of 3.6 mm² per grid cell was sufficient for detecting PE (as low as 1.6%) and PA (as low as 5.0%) in soils without the digestion and separation procedures.

Markedly Enhanced Analysis of Mass Spectrometry Images Using Weakly Supervised Machine Learning

Supervised and unsupervised machine learning algorithms are routinely applied to time-of-flight secondary ion mass spectrometry (ToF-SIMS) imaging data and, more broadly, to mass spectrometry imaging (MSI). These algorithms have accelerated large-scale, single-pixel analysis, classification, and regression. However, there is relatively little research on methods suited for so-called weakly supervised problems, where ground-truth class labels exist at the image level, but not at the individual pixel level. Unsupervised learning methods are usually applied to these problems. However, these methods cannot make use of available labels. Here a novel method specifically designed for weakly supervised MSI data is presented. A dual-stream multiple instance learning (MIL) approach is adapted from computational pathology that reveals the spatial-spectral characteristics distinguishing different classes of MSI images. The method uses an information entropy-regularized attention mechanism to identify characteristic class pixels that are then used to extract characteristic mass spectra. This work provides a proof-of-concept exemplification using printed ink samples imaged by ToF-SIMS. A second application-oriented study is also presented, focusing on the analysis of a mixed powder sample type. Results demonstrate the potential of the MIL method for broader application in MSI, with implications for understanding subtle spatial-spectral characteristics in various applications and contexts.

Machine learning extracts marks of thiamine's role in cold acclimation in the transcriptome of Vitis vinifera

Introduction

The escalating challenge of climate change has underscored the critical need to understand cold defense mechanisms in cultivated grapevine Vitis vinifera. Temperature variations can affect the growth and overall health of vine.

Methods

We used Self Organizing Maps machine learning method to analyze gene expression data from leaves of five Vitis vinifera cultivars each treated by four different temperature conditions. The algorithm generated sample-specific "portraits" of the normalized gene expression data, revealing distinct patterns related to the temperature conditions applied.

Results

Our analysis unveiled a connection with vitamin B1 (thiamine) biosynthesis, suggesting a link between temperature regulation and thiamine metabolism, in agreement with thiamine related stress response established in Arabidopsis before. Furthermore, we found that epigenetic mechanisms play a crucial role in regulating the expression of stress-responsive genes at low temperatures in grapevines.

Discussion

Application of Self Organizing Maps portrayal to vine transcriptomics identified modules of coregulated genes triggered under cold stress. Our machine learning approach provides a promising option for transcriptomics studies in plants.

Controllable hierarchical self-assembly: systematic study forming metallosupramolecular frameworks on the basis of helical beta-oligoamides

Self-assembly is a key guiding principle for the design of complex nanostructures. Substituted beta oligoamides offer versatile building blocks that can have inherent folding characteristics, offering geometrically defined functionalities that can specifically bind and assemble with predefined morphological characteristics. In this work hierarchical self-assembly is implemented based on metal coordinating helical beta-oligoamides crosslinked with transition metals selected for their favourable coordination geometries, Fe2+, Cu2+, Ni2+, Co2+, Zn2+, and two metalates, MoO42-, and WO42-. The oligoamide Ac-β3Aβ3Vβ3S-αHαHαH-β3Aβ3Vβ3A (3H) was designed to allow crosslinking via three distinct faces of the helical unit, with a possibility of forming three dimensional framework structures. Atomic force microscopy (AFM) confirmed the formation of specific morphologies that differ characteristically with each metal. X-Ray photoelectron spectroscopy (XPS) results reveal that the metal centres can be reduced in the final structures, confirming strong chemical interaction. Time of flight secondary ion mass spectrometry (ToF-SIMS) confirmed the spatial distribution of metals within the self-assembled networks, also revealing molecular fragments that confirm coordination to histidine and carboxyl moieties. The metalates MoO42- and WO42- were also able to induce the formation of specific superstructure morphologies. It was observed that assembly with either of nickel, copper, and molybdate form thin films, while cobalt, zinc, and tungstate produced specific three dimensional networks of oligoamides. Iron was found to form both a thin film and a complex hierarchical assembly with the 3H simultaneously. The design of the 3H substituted beta oligoamide to readily form metallosupramolecular frameworks was demonstrated with a range of metals and metalates with a degree of control over layer thicknesses as a function of the metal/metalate. The results validate and broaden the metallosupramolecular framework concept and establish a platform technology for the design of functional thin layer materials.

Multi-Dimensional Machine Learning Analysis of Polyaniline Films Using Stitched Hyperspectral ToF-SIMS Data

The self-organizing map with relational perspective mapping (SOM-RPM) is an unsupervised machine learning method that can be used to visualize and interpret high-dimensional hyperspectral data. We have previously used SOM-RPM for the analysis of time-of-flight secondary ion mass spectrometry (ToF-SIMS) hyperspectral images and three-dimensional (3D) depth profiles. This provides insightful visualization of features and trends of 3D depth profile data, using a slice-by-slice view, which can be useful for highlighting structural flaws including molecular characteristics and transport of contaminants to a buried interface and characterization of spectra. Here, we apply SOM-RPM to stitched ToF-SIMS data sets, whereby the stitched data are used to train the same model to provide a direct comparison in both 2D and 3D. We conduct an analysis of spin-coated polyaniline (PANI) films on indium tin oxide-coated glass slides that were subjected to heat treatment under atmospheric conditions to model PANI as a conformal aerospace industry coating. Replicates were shown to be precisely equivalent, both spatially and by composition, indicating a clear threshold for annealing of the film. Quantitative assessment was performed on the chemical breakdown trends accompanying annealing based on peak ratios, while spectral analysis alone shows only very subtle differences which are difficult to evaluate quantitatively. The SOM-RPM method considers data sets in their totality and highlights subtle differences between samples often simply differences in peak intensity ratios.

Hyperspectral Visualization-Based Mass Spectrometry Imaging by LMJ-SSP: A Novel Strategy for Rapid Natural Product Profiling in Bacteria

Mass spectrometry imaging (MSI) has been widely used to discover natural products (NPs) from underexplored microbiological sources. However, the technique is limited by incompatibility with complicated/uneven surface topography and labor-intensive sample preparation, as well as lengthy compound profiling procedures. Here, liquid micro-junction surface sampling probe (LMJ-SSP)-based MSI is used for rapid profiling of natural products from Gram-negative marine bacteria Pseudoalteromonas on nutrient agar media without any sample preparation. A conductance-based autosampling platform with 1 mm spatial resolution and an innovative multivariant analysis-driven method was used to create one hyperspectral image for the sampling area. NP discovery requires general spatial correlation between m/z and colony location but not highly precise spatial resolution. The hyperspectral image was used to annotate different m/z by straightforward color differences without the need to directly interrogate the spectra. To demonstrate the utility of our approach, the rapid analysis of Pseudoalteromonas rubra DSM6842, Pseudoalteromonas tunicata DSM14096, Pseudoalteromonas piscicida JCM20779, and Pseudoalteromonas elyakovii ATCC700519 cultures was directly performed on Agar. Various natural products, including prodiginine and tambjamine analogues, were quickly identified from the hyperspectral image, and the dynamic extracellular environment was shown with compound heatmaps. Hyperspectral visualization-based MSI is an efficient and sensitive strategy for direct and rapid natural product profiling from different Pseudoalteromonas strains.

Emerging Computational Methods in Mass Spectrometry Imaging

Mass spectrometry imaging (MSI) is a powerful analytical technique that generates maps of hundreds of molecules in biological samples with high sensitivity and molecular specificity. Advanced MSI platforms with capability of high-spatial resolution and high-throughput acquisition generate vast amount of data, which necessitates the development of computational tools for MSI data analysis. In addition, computation-driven MSI experiments have recently emerged as enabling technologies for further improving the MSI capabilities with little or no hardware modification. This review provides a critical summary of computational methods and resources developed for MSI data analysis and interpretation along with computational approaches for improving throughput and molecular coverage in MSI experiments. This review is focused on the recently developed artificial intelligence methods and provides an outlook for a future paradigm shift in MSI with transformative computational methods.

Information Entropy-Based Strategy for the Quantitative Evaluation of Extensive Hyperspectral Images to Better Unveil Spatial Heterogeneity in Mass Spectrometry Imaging

Hyperspectral images can be generated from mass spectrometry imaging (MSI) data for the intuitive data visualization purpose. However, hundreds of HSIs can be generated by different dimensionality reduction methods, which poses great challenges in selecting the high-quality images with the best intuitive visualization results of the MSI data. Here, we presented a novel approach that objectively evaluates the image quality of the hyperspectral images. The applicability of this method was demonstrated by analyzing the MSI data acquired from human prostate cancer biopsy samples and mouse brain tissue section, which harbored an intrinsic tissue heterogeneity. Our method was based on the information entropy and contrast measured from image information content and image definition, respectively. The heterogeneity of the MSI data from high-dimensional space was reduced to three-dimensional embeddings and thoroughly evaluated to achieve satisfactory visualization results. The application of information entropy and contrast can be used to choose the optimized visualization results rapidly and objectively from an extensive number of hyperspectral images and be adopted to evaluate and optimize different dimensionality reduction algorithms and their hyperparameter combinations. In conclusion, the information entropy-based strategy could be a bridge between chemometrician and biologists.

Two-Dimensional and Three-Dimensional Time-of-Flight Secondary Ion Mass Spectrometry Image Feature Extraction Using a Spatially Aware Convolutional Autoencoder

Feature extraction algorithms are an important class of unsupervised methods used to reduce data dimensionality. They have been applied extensively for time-of-flight secondary ion mass spectrometry (ToF-SIMS) imaging─commonly, matrix factorization (MF) techniques such as principal component analysis have been used. A limitation of MF is the assumption of linearity, which is generally not accurate for ToF-SIMS data. Recently, nonlinear autoencoders have been shown to outperform MF techniques for ToF-SIMS image feature extraction. However, another limitation of most feature extraction methods (including autoencoders) that is particularly important for hyperspectral data is that they do not consider spatial information. To address this limitation, we describe the application of the convolutional autoencoder (CNNAE) to hyperspectral ToF-SIMS imaging data. The CNNAE is an artificial neural network developed specifically for hyperspectral data that uses convolutional layers for image encoding, thereby explicitly incorporating pixel neighborhood information. We compared the performance of the CNNAE with other common feature extraction algorithms for two biological ToF-SIMS imaging data sets. We investigated the extracted features and used the dimensionality-reduced data to train additional ML algorithms. By converting two-dimensional convolutional layers to three-dimensional (3D), we also showed how the CNNAE can be extended to 3D ToF-SIMS images. In general, the CNNAE produced features with significantly higher contrast and autocorrelation than other techniques. Furthermore, histologically recognizable features in the data were more accurately represented. The extension of the CNNAE to 3D data also provided an important proof of principle for the analysis of more complex 3D data sets.

Nonutilization of Kidneys From Donors After Circulatory Determinant of Death

Background.

The expansion of donation after circulatory determination of death (DCDD) programs and unmet demands for kidney transplantation indicate that there is a need to improve the efficiency and utilization of these organs.

Methods.

We studied all DCDD donors retrieved for kidney transplantation in Australia between 2014 and 2019 and determined the factors associated with nonutilization using least absolute shrinkage and selection operator and random forest models. Self-organizing maps were used to group these donors into clusters with similar characteristics and features associated with nonutilization were defined.

Results.

Of the 762 DCDD donors, 116 (15%) were not utilized for kidney transplantation. Of the 9 clusters derived from self-organizing map, 2 had the highest proportions of nonutilized kidneys. Factors for nonutilization (adjusted odds ratio [95% confidence interval], per SD increase) were duration from withdrawal of cardiorespiratory support till death (1.38 [1.16-1.64]), admission and terminal serum creatinine (1.43 [1.13-1.85]) and (1.41 [1.16-1.73]). Donor kidney function and duration of warm ischemia were the main factors for clinical decisions taken not to use kidneys from DCDD donors.

Conclusions.

Donor terminal kidney function and the duration of warm ischemia are the key factors for nonutilization of DCDD kidneys. Strategies to reduce the duration of warm ischemia and improve post-transplant recipient kidney function may reduce rates of nonutilization.

A new hybrid ensemble machine-learning model for severity risk assessment and post-COVID prediction system

Starting from December 2019, the COVID-19 pandemic has globally strained medical resources and caused significant mortality. It is commonly recognized that the severity of SARS-CoV-2 disease depends on both the comorbidity and the state of the patient's immune system, which is reflected in several biomarkers. The development of early diagnosis and disease severity prediction methods can reduce the burden on the health care system and increase the effectiveness of treatment and rehabilitation of patients with severe cases. This study aims to develop and validate an ensemble machine-learning model based on clinical and immunological features for severity risk assessment and post-COVID rehabilitation duration for SARS-CoV-2 patients. The dataset consisting of 35 features and 122 instances was collected from Lviv regional rehabilitation center. The dataset contains age, gender, weight, height, BMI, CAT, 6-minute walking test, pulse, external respiration function, oxygen saturation, and 15 immunological markers used to predict the relationship between disease duration and biomarkers using the machine learning approach. The predictions are assessed through an area under the receiver-operating curve, classification accuracy, precision, recall, and F1 score performance metrics. A new hybrid ensemble feature selection model for a post-COVID prediction system is proposed as an automatic feature cut-off rank identifier. A three-layer high accuracy stacking ensemble classification model for intelligent analysis of short medical datasets is presented. Together with weak predictors, the associative rules allowed improving the classification quality. The proposed ensemble allows using a random forest model as an aggregator for weak repressors' results generalization. The performance of the three-layer stacking ensemble classification model (AUC 0.978; CA 0.920; F1 score 0.921; precision 0.924; recall 0.920) was higher than five machine learning models, viz. tree algorithm with forward pruning; Naïve Bayes classifier; support vector machine with RBF kernel; logistic regression, and a calibrated learner with sigmoid function and decision threshold optimization. Aging-related biomarkers, viz. CD3+, CD4+, CD8+, CD22+ were examined to predict post-COVID rehabilitation duration. The best accuracy was reached in the case of the support vector machine with the linear kernel (MAPE = 0.0787) and random forest classifier (RMSE = 1.822). The proposed three-layer stacking ensemble classification model predicted SARS-CoV-2 disease severity based on the cytokines and physiological biomarkers. The results point out that changes in studied biomarkers associated with the severity of the disease can be used to monitor the severity and forecast the rehabilitation duration.

Applications of multivariate analysis and unsupervised machine learning to ToF-SIMS images of organic, bioorganic, and biological systems

Time-of-flight secondary ion mass spectrometry (ToF-SIMS) imaging offers a powerful, label-free method for exploring organic, bioorganic, and biological systems. The technique is capable of very high spatial resolution, while also producing an enormous amount of information about the chemical and molecular composition of a surface. However, this information is inherently complex, making interpretation and analysis of the vast amount of data produced by a single ToF-SIMS experiment a considerable challenge. Much research over the past few decades has focused on the application and development of multivariate analysis (MVA) and machine learning (ML) techniques that find meaningful patterns and relationships in these datasets. Here, we review the unsupervised algorithms—that is, algorithms that do not require ground truth labels—that have been applied to ToF-SIMS images, as well as other algorithms and approaches that have been used in the broader family of mass spectrometry imaging (MSI) techniques. We first give a nontechnical overview of several commonly used classes of unsupervised algorithms, such as matrix factorization, clustering, and nonlinear dimensionality reduction. We then review the application of unsupervised algorithms to various organic, bioorganic, and biological systems including cells and tissues, organic films, residues and coatings, and spatially structured systems such as polymer microarrays. We then cover several novel algorithms employed for other MSI techniques that have received little attention from ToF-SIMS imaging researchers. We conclude with a brief outline of potential future directions for the application of MVA and ML algorithms to ToF-SIMS images.

Revealing Contamination and Sequence of Overlapping Fingerprints by Unsupervised Treatment of a Hyperspectral Secondary Ion Mass Spectrometry Dataset

Time-of-flight secondary ion mass spectrometry (ToF-SIMS) has been successfully applied for chemical imaging of overlapping fingermarks. The resulting big dataset has been treated by means of an unsupervised machine learning approach based on uniform manifold approximation and projection. The hyperspectral matrix was composed of 49 million pixels associated with 518 peaks. However, the single-pixel spectrum results in a very poor signal intensity, mostly like a barcode. Contrary to what has been reported in the literature recently, we have not applied a crude approach based on binning but a sophisticated machine learning method capable of separating the chemical signals of the two fingerprints from each other and from the substrate in which they were impressed. Moreover, using ToF-SIMS, an extremely surface-sensitive technique, the sequence of deposition of the fingerprints has been determined.

Spatial Segmentation of Mass Spectrometry Imaging Data by Combining Multivariate Clustering and Univariate Thresholding

Spatial segmentation partitions mass spectrometry imaging (MSI) data into distinct regions, providing a concise visualization of the vast amount of data and identifying regions of interest (ROIs) for downstream statistical analysis. Unsupervised approaches are particularly attractive, as they may be used to discover the underlying subpopulations present in the high-dimensional MSI data without prior knowledge of the properties of the sample. Herein, we introduce an unsupervised spatial segmentation approach, which combines multivariate clustering and univariate thresholding to generate comprehensive spatial segmentation maps of the MSI data. This approach combines matrix factorization and manifold learning to enable high-quality image segmentation without an extensive hyperparameter search. In parallel, some ion images inadequately represented in the multivariate analysis were treated using univariate thresholding to generate complementary spatial segments. The final spatial segmentation map was assembled from segment candidates that were generated using both techniques. We demonstrate the performance and robustness of this approach for two MSI data sets of mouse uterine and kidney tissue sections that were acquired with different spatial resolutions. The resulting segmentation maps are easy to interpret and project onto the known anatomical regions of the tissue.

Analyzing 3D hyperspectral TOF-SIMS depth profile data using self-organizing map-relational perspective mapping

The advantages of applying multivariate analysis to mass spectrometry imaging (MSI) data have been thoroughly demonstrated in recent decades. The identification and visualization of complex relationships between pixels in a hyperspectral data set can provide unique insights into the underlying surface chemistry. It is now recognized that most MSI data contain nonlinear relationships, which has led to increased application of machine learning approaches. Previously, we exemplified the use of the self-organizing map (SOM), a type of artificial neural network, for analyzing time-of-flight secondary ion mass spectrometry (TOF-SIMS) hyperspectral images. Recently, we developed a novel methodology, SOM-relational perspective mapping (RPM), which incorporates the algorithm RPM to improve visualization of the SOM for 2D TOF-SIMS images. Here, we use SOM-RPM to characterize and interpret 3D TOF-SIMS depth profile data, voxel-by-voxel. An organic Irganox^™ multilayer standard sample was depth profiled using TOF-SIMS, and SOM-RPM was used to create 3D similarity maps of the depth-profiled sample, in which the mass spectral similarity of individual voxels is modeled with color similarity. We used this similarity map to segment the data into spatial features, demonstrating that the unsupervised method meaningfully differentiated between Irganox-3114 and Irganox-1010 nanometer-thin multilayer films. The method also identified unique clusters at the surface associated with environmental exposure and sample degradation. Key fragment ions characteristic of each cluster were identified, tying clusters to their underlying chemistries. SOM-RPM has the demonstrable ability to reduce vast data sets to simple 3D visualizations that can be used for clustering data and visualizing the complex relationships within.

Understanding mass spectrometry images: complexity to clarity with machine learning

The application of artificial intelligence and machine learning to hyperspectral mass spectrometry imaging (MSI) data has received considerable attention over recent years. Various methodologies have shown great promise in their ability to handle the complexity and size of MSI data sets. Advances in this area have been particularly appealing for MSI of biological samples, which typically produce highly complicated data with often subtle relationships between features. There are many different machine learning approaches that have been applied to MSI data over the past two decades. In this review, we focus on a subset of non-linear machine learning techniques that have mostly only been applied in the past 5 years. Specifically, we review the use of the self-organizing map (SOM), SOM with relational perspective mapping (SOM-RPM), t-distributed stochastic neighbor embedding (t-SNE) and uniform manifold approximation and projection (UMAP). While not their only functionality, we have grouped these techniques based on their ability to produce what we refer to as similarity maps. Similarity maps are color representations of hyperspectral data, in which spectral similarity between pixels-that is, their distance in high-dimensional space-is represented by relative color similarity. In discussing these techniques, we describe, briefly, their associated algorithms and functionalities, and also outline applications in MSI research with a strong focus on biological sample types. The aim of this review is therefore to introduce this relatively recent paradigm for visualizing and exploring hyperspectral MSI, while also providing a comparison between each technique discussed.