Spatially aware clustering of ion images in mass spectrometry imaging data using deep learning

[Research progress of deep learning applications in mass spectrometry imaging data analysis]

Mass spectrometry imaging (MSI) is a promising method for characterizing the spatial distribution of compounds. Given the diversified development of acquisition methods and continuous improvements in the sensitivity of this technology, both the total amount of generated data and complexity of analysis have exponentially increased, rendering increasing challenges of data postprocessing, such as large amounts of noise, background signal interferences, as well as image registration deviations caused by sample position changes and scan deviations, and etc. Deep learning (DL) is a powerful tool widely used in data analysis and image reconstruction. This tool enables the automatic feature extraction of data by building and training a neural network model, and achieves comprehensive and in-depth analysis of target data through transfer learning, which has great potential for MSI data analysis. This paper reviews the current research status, application progress and challenges of DL in MSI data analysis, focusing on four core stages: data preprocessing, image reconstruction, cluster analysis, and multimodal fusion. The application of a combination of DL and mass spectrometry imaging in the study of tumor diagnosis and subtype classification is also illustrated. This review also discusses trends of development in the future, aiming to promote a better combination of artificial intelligence and mass spectrometry technology.

Recent Developments in Machine Learning for Mass Spectrometry

Statistical analysis and modeling of mass spectrometry (MS) data have a long and rich history with several modern MS-based applications using statistical and chemometric methods. Recently, machine learning (ML) has experienced a renaissance due to advents in computational hardware and the development of new algorithms for artificial neural networks (ANN) and deep learning architectures. Moreover, recent successes of new ANN and deep learning architectures in several areas of science, engineering, and society have further strengthened the ML field. Importantly, modern ML methods and architectures have enabled new approaches for tasks related to MS that are now widely adopted in several popular MS-based subdisciplines, such as mass spectrometry imaging and proteomics. Herein, we aim to provide an introductory summary of the practical aspects of ML methodology relevant to MS. Additionally, we seek to provide an up-to-date review of the most recent developments in ML integration with MS-based techniques while also providing critical insights into the future direction of the field.

Modern machine-learning applications in ambient ionization mass spectrometry

This article provides a comprehensive overview of the applications of methods of machine learning (ML) and artificial intelligence (AI) in ambient ionization mass spectrometry (AIMS). AIMS has emerged as a powerful analytical tool in recent years, allowing for rapid and sensitive analysis of various samples without the need for extensive sample preparation. The integration of ML/AI algorithms with AIMS has further expanded its capabilities, enabling enhanced data analysis. This review discusses ML/AI algorithms applicable to the AIMS data and highlights the key advancements and potential benefits of utilizing ML/AI in the field of mass spectrometry, with a focus on the AIMS community.

Mass spectrometry imaging methods for visualizing tumor heterogeneity

Profiling spatial distributions of lipids, metabolites, and proteins in tumors can reveal unique cellular microenvironments and provide molecular evidence for cancer cell dysfunction and proliferation. Mass spectrometry imaging (MSI) is a label-free technique that can be used to map biomolecules in tumors in situ. Here, we discuss current progress in applying MSI to uncover molecular heterogeneity in tumors. First, the analytical strategies to profile small molecules and proteins are outlined, and current methods for multimodal imaging to maximize biological information are highlighted. Second, we present and summarize biological insights obtained by MSI of tumor tissue. Finally, we discuss important considerations for designing MSI experiments and several current analytical challenges.

A Machine Learning-Driven Comparison of Ion Images Obtained by MALDI and MALDI-2 Mass Spectrometry Imaging

Matrix-assisted laser desorption ionization mass spectrometry imaging (MALDI-MSI) enables label-free imaging of biomolecules in biological tissues. However, many species remain undetected due to their poor ionization efficiencies. MALDI-2 (laser-induced post-ionization) is the most widely used post-ionization method for improving analyte ionization efficiencies. Mass spectra acquired using MALDI-2 constitute a combination of ions generated by both MALDI and MALDI-2 processes. Until now, no studies have focused on a detailed comparison between the ion images (as opposed to the generated m/z values) produced by MALDI and MALDI-2 for mass spectrometry imaging (MSI) experiments. Herein, we investigated the ion images produced by both MALDI and MALDI-2 on the same tissue section using correlation analysis (to explore similarities in ion images for ions common to both MALDI and MALDI-2) and a deep learning approach. For the latter, we used an analytical workflow based on the Xception convolutional neural network, which was originally trained for human-like natural image classification but which we adapted to elucidate similarities and differences in ion images obtained using the two MSI techniques. Correlation analysis demonstrated that common ions yielded similar spatial distributions with low-correlation species explained by either poor signal intensity in MALDI or the generation of additional unresolved signals using MALDI-2. Using the Xception-based method, we identified many regions in the t-SNE space of spatially similar ion images containing MALDI and MALDI-2-related signals. More notably, the method revealed distinct regions containing only MALDI-2 ion images with unique spatial distributions that were not observed using MALDI. These data explicitly demonstrate the ability of MALDI-2 to reveal molecular features and patterns as well as histological regions of interest that are not visible when using conventional MALDI.

DeepION: A Deep Learning-Based Low-Dimensional Representation Model of Ion Images for Mass Spectrometry Imaging

Mass spectrometry imaging (MSI) is a high-throughput imaging technique capable of the qualitative and quantitative in situ detection of thousands of ions in biological samples. Ion image representation is a technique that produces a low-dimensional vector embedded with significant spectral and spatial information on an ion image, which further facilitates the distance-based similarity measurement for the identification of colocalized ions. However, given the low signal-to-noise ratios inherent in MSI data coupled with the scarcity of annotated data sets, achieving an effective ion image representation for each ion image remains a challenge. In this study, we propose DeepION, a novel deep learning-based method designed specifically for ion image representation, which is applied to the identification of colocalized ions and isotope ions. In DeepION, contrastive learning is introduced to ensure that the model can generate the ion image representation in a self-supervised manner without manual annotation. Since data augmentation is a crucial step in contrastive learning, a unique data augmentation strategy is designed by considering the characteristics of MSI data, such as the Poisson distribution of ion abundance and a random pattern of missing values, to generate plentiful ion image pairs for DeepION model training. Experimental results of rat brain tissue MSI show that DeepION outperforms other methods for both colocalized ion and isotope ion identification, demonstrating the effectiveness of ion image representation. The proposed model could serve as a crucial tool in the biomarker discovery and drug development of the MSI technique.

rMSIfragment: improving MALDI-MSI lipidomics through automated in-source fragment annotation

Matrix-Assisted Laser Desorption Ionization Mass Spectrometry Imaging (MALDI-MSI) spatially resolves the chemical composition of tissues. Lipids are of particular interest, as they influence important biological processes in health and disease. However, the identification of lipids in MALDI-MSI remains a challenge due to the lack of chromatographic separation or untargeted tandem mass spectrometry. Recent studies have proposed the use of MALDI in-source fragmentation to infer structural information and aid identification. Here we present rMSIfragment, an open-source R package that exploits known adducts and fragmentation pathways to confidently annotate lipids in MALDI-MSI. The annotations are ranked using a novel score that demonstrates an area under the curve of 0.7 in ROC analyses using HPLC-MS and Target-Decoy validations. rMSIfragment applies to multiple MALDI-MSI sample types and experimental setups. Finally, we demonstrate that overlooking in-source fragments increases the number of incorrect annotations. Annotation workflows should consider in-source fragmentation tools such as rMSIfragment to increase annotation confidence and reduce the number of false positives.

Prospective on Imaging Mass Spectrometry in Clinical Diagnostics

Imaging mass spectrometry (IMS) is a molecular technology utilized for spatially driven research, providing molecular maps from tissue sections. This article reviews matrix-assisted laser desorption ionization (MALDI) IMS and its progress as a primary tool in the clinical laboratory. MALDI mass spectrometry has been used to classify bacteria and perform other bulk analyses for plate-based assays for many years. However, the clinical application of spatial data within a tissue biopsy for diagnoses and prognoses is still an emerging opportunity in molecular diagnostics. This work considers spatially driven mass spectrometry approaches for clinical diagnostics and addresses aspects of new imaging-based assays that include analyte selection, quality control/assurance metrics, data reproducibility, data classification, and data scoring. It is necessary to implement these tasks for the rigorous translation of IMS to the clinical laboratory; however, this requires detailed standardized protocols for introducing IMS into the clinical laboratory to deliver reliable and reproducible results that inform and guide patient care.

Spatial Proteomics for the Molecular Characterization of Breast Cancer

Breast cancer (BC) is a major global health issue, affecting a significant proportion of the female population and contributing to high rates of mortality. One of the primary challenges in the treatment of BC is the disease's heterogeneity, which can lead to ineffective therapies and poor patient outcomes. Spatial proteomics, which involves the study of protein localization within cells, offers a promising approach for understanding the biological processes that contribute to cellular heterogeneity within BC tissue. To fully leverage the potential of spatial proteomics, it is critical to identify early diagnostic biomarkers and therapeutic targets, and to understand protein expression levels and modifications. The subcellular localization of proteins is a key factor in their physiological function, making the study of subcellular localization a major challenge in cell biology. Achieving high resolution at the cellular and subcellular level is essential for obtaining an accurate spatial distribution of proteins, which in turn can enable the application of proteomics in clinical research. In this review, we present a comparison of current methods of spatial proteomics in BC, including untargeted and targeted strategies. Untargeted strategies enable the detection and analysis of proteins and peptides without a predetermined molecular focus, whereas targeted strategies allow the investigation of a predefined set of proteins or peptides of interest, overcoming the limitations associated with the stochastic nature of untargeted proteomics. By directly comparing these methods, we aim to provide insights into their strengths and limitations and their potential applications in BC research.

Systems Immunology Approaches to Metabolism

Over the last decade, immunometabolism has emerged as a novel interdisciplinary field of research and yielded significant fundamental insights into the regulation of immune responses. Multiple classical approaches to interrogate immunometabolism, including bulk metabolic profiling and analysis of metabolic regulator expression, paved the way to appreciating the physiological complexity of immunometabolic regulation in vivo. Studying immunometabolism at the systems level raised the need to transition towards the next-generation technology for metabolic profiling and analysis. Spatially resolved metabolic imaging and computational algorithms for multi-modal data integration are new approaches to connecting metabolism and immunity. In this review, we discuss recent studies that highlight the complex physiological interplay between immune responses and metabolism and give an overview of technological developments that bear the promise of capturing this complexity most directly and comprehensively.

A noise-robust deep clustering of biomolecular ions improves interpretability of mass spectrometric images

MOTIVATION

Mass Spectrometry Imaging (MSI) analyzes complex biological samples such as tissues. It simultaneously characterizes the ions present in the tissue in the form of mass spectra, and the spatial distribution of the ions across the tissue in the form of ion images. Unsupervised clustering of ion images facilitates the interpretation in the spectral domain, by identifying groups of ions with similar spatial distributions. Unfortunately, many current methods for clustering ion images ignore the spatial features of the images, and are therefore unable to learn these features for clustering purposes. Alternative methods extract spatial features using deep neural networks pre-trained on natural image tasks; however, this is often inadequate since ion images are substantially noisier than natural images.

RESULTS

We contribute a deep clustering approach for ion images that accounts for both spatial contextual features and noise. In evaluations on a simulated dataset and on four experimental datasets of different tissue types, the proposed method grouped ions from the same source into a same cluster more frequently than existing methods. We further demonstrated that using ion image clustering as a pre-processing step facilitated the interpretation of a subsequent spatial segmentation as compared to using either all the ions or one ion at a time. As a result, the proposed approach facilitated the interpretability of MSI data in both the spectral domain and the spatial domain.

AVAILABILITYAND IMPLEMENTATION

The data and code are available at https://github.com/DanGuo1223/mzClustering.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

DiviK: divisive intelligent K-means for hands-free unsupervised clustering in big biological data

BACKGROUND

Investigating molecular heterogeneity provides insights into tumour origin and metabolomics. The increasing amount of data gathered makes manual analyses infeasible-therefore, automated unsupervised learning approaches are utilised for discovering tissue heterogeneity. However, automated analyses require experience setting the algorithms' hyperparameters and expert knowledge about the analysed biological processes. Moreover, feature engineering is needed to obtain valuable results because of the numerous features measured.

RESULTS

We propose DiviK: a scalable stepwise algorithm with local data-driven feature space adaptation for segmenting high-dimensional datasets. The algorithm is compared to the optional solutions (regular k-means, spatial and spectral approaches) combined with different feature engineering techniques (None, PCA, EXIMS, UMAP, Neural Ions). Three quality indices: Dice Index, Rand Index and EXIMS score, focusing on the overall composition of the clustering, coverage of the tumour region and spatial cluster consistency, are used to assess the quality of unsupervised analyses. Algorithms were validated on mass spectrometry imaging (MSI) datasets-2D human cancer tissue samples and 3D mouse kidney images. DiviK algorithm performed the best among the four clustering algorithms compared (overall quality score 1.24, 0.58 and 162 for d(0, 0, 0), d(1, 1, 1) and the sum of ranks, respectively), with spectral clustering being mostly second. Feature engineering techniques impact the overall clustering results less than the algorithms themselves (partial [Formula: see text] effect size: 0.141 versus 0.345, Kendall's concordance index: 0.424 versus 0.138 for d(0, 0, 0)).

CONCLUSIONS

DiviK could be the default choice in the exploration of MSI data. Thanks to its unique, GMM-based local optimisation of the feature space and deglomerative schema, DiviK results do not strongly depend on the feature engineering technique applied and can reveal the hidden structure in a tissue sample. Additionally, DiviK shows high scalability, and it can process at once the big omics data with more than 1.5 mln instances and a few thousand features. Finally, due to its simplicity, DiviK is easily generalisable to an even more flexible framework. Therefore, it is helpful for other -omics data (as single cell spatial transcriptomic) or tabular data in general (including medical images after appropriate embedding). A generic implementation is freely available under Apache 2.0 license at https://github.com/gmrukwa/divik .

Emerging Computational Methods in Mass Spectrometry Imaging

Mass spectrometry imaging (MSI) is a powerful analytical technique that generates maps of hundreds of molecules in biological samples with high sensitivity and molecular specificity. Advanced MSI platforms with capability of high-spatial resolution and high-throughput acquisition generate vast amount of data, which necessitates the development of computational tools for MSI data analysis. In addition, computation-driven MSI experiments have recently emerged as enabling technologies for further improving the MSI capabilities with little or no hardware modification. This review provides a critical summary of computational methods and resources developed for MSI data analysis and interpretation along with computational approaches for improving throughput and molecular coverage in MSI experiments. This review is focused on the recently developed artificial intelligence methods and provides an outlook for a future paradigm shift in MSI with transformative computational methods.

Identifying multicellular spatiotemporal organization of cells with SpaceFlow

One major challenge in analyzing spatial transcriptomic datasets is to simultaneously incorporate the cell transcriptome similarity and their spatial locations. Here, we introduce SpaceFlow, which generates spatially-consistent low-dimensional embeddings by incorporating both expression similarity and spatial information using spatially regularized deep graph networks. Based on the embedding, we introduce a pseudo-Spatiotemporal Map that integrates the pseudotime concept with spatial locations of the cells to unravel spatiotemporal patterns of cells. By comparing with multiple existing methods on several spatial transcriptomic datasets at both spot and single-cell resolutions, SpaceFlow is shown to produce a robust domain segmentation and identify biologically meaningful spatiotemporal patterns. Applications of SpaceFlow reveal evolving lineage in heart developmental data and tumor-immune interactions in human breast cancer data. Our study provides a flexible deep learning framework to incorporate spatiotemporal information in analyzing spatial transcriptomic data.

A critical task in spatial transcriptomics analysis is to understand inherently spatial relationships among cells. Here, the authors present a deep learning framework to integrate spatial and transcriptional information, spatially extending pseudotime and revealing spatiotemporal organization of cells.

What are we imaging? Software tools and experimental strategies for annotation and identification of small molecules in mass spectrometry imaging

Mass spectrometry imaging (MSI) has become a widespread analytical technique to perform nonlabeled spatial molecular identification. The Achilles' heel of MSI is the annotation and identification of molecular species due to intrinsic limitations of the technique (lack of chromatographic separation and the difficulty to apply tandem MS). Successful strategies to perform annotation and identification combine extra analytical steps, like using orthogonal analytical techniques to identify compounds; with algorithms that integrate the spectral and spatial information. In this review, we discuss different experimental strategies and bioinformatics tools to annotate and identify compounds in MSI experiments. We target strategies and tools for small molecule applications, such as lipidomics and metabolomics. First, we explain how sample preparation and the acquisition process influences annotation and identification, from sample preservation to the use of orthogonal techniques. Then, we review twelve software tools for annotation and identification in MSI. Finally, we offer perspectives on two current needs of the MSI community: the adaptation of guidelines for communicating confidence levels in identifications; and the creation of a standard format to store and exchange annotations and identifications in MSI.

Two-Dimensional and Three-Dimensional Time-of-Flight Secondary Ion Mass Spectrometry Image Feature Extraction Using a Spatially Aware Convolutional Autoencoder

Feature extraction algorithms are an important class of unsupervised methods used to reduce data dimensionality. They have been applied extensively for time-of-flight secondary ion mass spectrometry (ToF-SIMS) imaging─commonly, matrix factorization (MF) techniques such as principal component analysis have been used. A limitation of MF is the assumption of linearity, which is generally not accurate for ToF-SIMS data. Recently, nonlinear autoencoders have been shown to outperform MF techniques for ToF-SIMS image feature extraction. However, another limitation of most feature extraction methods (including autoencoders) that is particularly important for hyperspectral data is that they do not consider spatial information. To address this limitation, we describe the application of the convolutional autoencoder (CNNAE) to hyperspectral ToF-SIMS imaging data. The CNNAE is an artificial neural network developed specifically for hyperspectral data that uses convolutional layers for image encoding, thereby explicitly incorporating pixel neighborhood information. We compared the performance of the CNNAE with other common feature extraction algorithms for two biological ToF-SIMS imaging data sets. We investigated the extracted features and used the dimensionality-reduced data to train additional ML algorithms. By converting two-dimensional convolutional layers to three-dimensional (3D), we also showed how the CNNAE can be extended to 3D ToF-SIMS images. In general, the CNNAE produced features with significantly higher contrast and autocorrelation than other techniques. Furthermore, histologically recognizable features in the data were more accurately represented. The extension of the CNNAE to 3D data also provided an important proof of principle for the analysis of more complex 3D data sets.

Applications of multivariate analysis and unsupervised machine learning to ToF-SIMS images of organic, bioorganic, and biological systems

Time-of-flight secondary ion mass spectrometry (ToF-SIMS) imaging offers a powerful, label-free method for exploring organic, bioorganic, and biological systems. The technique is capable of very high spatial resolution, while also producing an enormous amount of information about the chemical and molecular composition of a surface. However, this information is inherently complex, making interpretation and analysis of the vast amount of data produced by a single ToF-SIMS experiment a considerable challenge. Much research over the past few decades has focused on the application and development of multivariate analysis (MVA) and machine learning (ML) techniques that find meaningful patterns and relationships in these datasets. Here, we review the unsupervised algorithms—that is, algorithms that do not require ground truth labels—that have been applied to ToF-SIMS images, as well as other algorithms and approaches that have been used in the broader family of mass spectrometry imaging (MSI) techniques. We first give a nontechnical overview of several commonly used classes of unsupervised algorithms, such as matrix factorization, clustering, and nonlinear dimensionality reduction. We then review the application of unsupervised algorithms to various organic, bioorganic, and biological systems including cells and tissues, organic films, residues and coatings, and spatially structured systems such as polymer microarrays. We then cover several novel algorithms employed for other MSI techniques that have received little attention from ToF-SIMS imaging researchers. We conclude with a brief outline of potential future directions for the application of MVA and ML algorithms to ToF-SIMS images.

A Neural Network-Based Method for Fast Capture and Tracking of Laser Links between Nonorbiting Platforms

In this paper, a neural network approach is used to conduct an in-depth study and analysis of the fast capture tracking method for laser links between nonorbiting platforms. The experimental platform of the convolutional neural network- (CNN-) based free-space optical communication (FSO) wavefront correction system is built indoors, and the wavefront distortion correction performance of the CNN-based wavefront correction method is investigated. The experimental results show that the coupling power loss can be reduced to small after the CNN method correction under weak and strong turbulence. The accuracy of the above model is verified by comparing the simulation data with the experimentally measured data, thus realizing the coordinate decoupling of the coarse aiming mechanism and weakening the influence of structural factors on the tracking accuracy of the system. The tracking correlation equation of the influence of beam far-field dynamic characteristics on the tracking stability of the link is established, and the correlation factor variance of beam far-field dynamic characteristics is used to provide a quantitative analysis method for the evaluation and prediction of the comprehensive performance of the link tracking stability. The influence of beam divergence angle, wavefront distortion, detector accuracy, and atmospheric turbulence disturbance on the correlation factor variance of beam far-field dynamic characteristics of laser link beacons is modelled, and the link tracking stability optimization method is proposed under the requirement of link tracking accuracy, which provides an effective solution analysis method to realize the improvement of laser link tracking stability.

Data parsing in mass spectrometry imaging using R Studio and Cardinal: A tutorial

Mass spectrometry imaging (MSI) has emerged as a rapidly expanding field in the MS community. The analysis of large spectral data is further complicated by the added spatial dimension of MSI. A plethora of resources exist for expert users to begin parsing MSI data in R, but there is a critical lack of guidance for absolute beginners. This tutorial is designed to serve as a one-stop guide to start using R with MSI data and describe the possibilities that data science can bring to MSI analysis.

Self-supervised clustering of mass spectrometry imaging data using contrastive learning

Mass spectrometry imaging (MSI) is widely used for the label-free molecular mapping of biological samples. The identification of co-localized molecules in MSI data is crucial to the understanding of biochemical pathways. One of key challenges in molecular colocalization is that complex MSI data are too large for manual annotation but too small for training deep neural networks. Herein, we introduce a self-supervised clustering approach based on contrastive learning, which shows an excellent performance in clustering of MSI data. We train a deep convolutional neural network (CNN) using MSI data from a single experiment without manual annotations to effectively learn high-level spatial features from ion images and classify them based on molecular colocalizations. We demonstrate that contrastive learning generates ion image representations that form well-resolved clusters. Subsequent self-labeling is used to fine-tune both the CNN encoder and linear classifier based on confidently classified ion images. This new approach enables autonomous and high-throughput identification of co-localized species in MSI data, which will dramatically expand the application of spatial lipidomics, metabolomics, and proteomics in biological research.