Deep learning for tumor classification in imaging mass spectrometry

Mass spectrometry imaging for spatially resolved multi-omics molecular mapping

The recent upswing in the integration of spatial multi-omics for conducting multidimensional information measurements is opening a new chapter in biological research. Mapping the landscape of various biomolecules including metabolites, proteins, nucleic acids, etc., and even deciphering their functional interactions and pathways is believed to provide a more holistic and nuanced exploration of the molecular intricacies within living systems. Mass spectrometry imaging (MSI) stands as a forefront technique for spatially mapping the metabolome, lipidome, and proteome within diverse tissue and cell samples. In this review, we offer a systematic survey delineating different MSI techniques for spatially resolved multi-omics analysis, elucidating their principles, capabilities, and limitations. Particularly, we focus on the advancements in methodologies aimed at augmenting the molecular sensitivity and specificity of MSI; and depict the burgeoning integration of MSI-based spatial metabolomics, lipidomics, and proteomics, encompassing the synergy with other imaging modalities. Furthermore, we offer speculative insights into the potential trajectory of MSI technology in the future.

Markedly Enhanced Analysis of Mass Spectrometry Images Using Weakly Supervised Machine Learning

Supervised and unsupervised machine learning algorithms are routinely applied to time-of-flight secondary ion mass spectrometry (ToF-SIMS) imaging data and, more broadly, to mass spectrometry imaging (MSI). These algorithms have accelerated large-scale, single-pixel analysis, classification, and regression. However, there is relatively little research on methods suited for so-called weakly supervised problems, where ground-truth class labels exist at the image level, but not at the individual pixel level. Unsupervised learning methods are usually applied to these problems. However, these methods cannot make use of available labels. Here a novel method specifically designed for weakly supervised MSI data is presented. A dual-stream multiple instance learning (MIL) approach is adapted from computational pathology that reveals the spatial-spectral characteristics distinguishing different classes of MSI images. The method uses an information entropy-regularized attention mechanism to identify characteristic class pixels that are then used to extract characteristic mass spectra. This work provides a proof-of-concept exemplification using printed ink samples imaged by ToF-SIMS. A second application-oriented study is also presented, focusing on the analysis of a mixed powder sample type. Results demonstrate the potential of the MIL method for broader application in MSI, with implications for understanding subtle spatial-spectral characteristics in various applications and contexts.

Deep learning-assisted mass spectrometry imaging for preliminary screening and pre-classification of psychoactive substances

Currently, it is of great urgency to develop a rapid pre-classification and screening method for suspected drugs as the constantly springing up of new psychoactive substances. In most researches, psychoactive substances classification approaches depended on the similar chemical structures and pharmacological action with known drugs. Such approaches could not face the complicated circumstance of emerging new psychoactive substances. Herein, mass spectrometry imaging and convolutional neural networks (CNN) were used for preliminary screening and pre-classification of suspected psychoactive substances. Mass spectrometry imaging was performed simultaneously on two brain slices as one was from blank group and another one was from psychoactive substance-induced group. Then, fused neurotransmitter variation mass spectrometry images (Nv-MSIs) reflecting the difference of neurotransmitters between two slices were achieved through two homemade programs. A CNN model was developed to classify the Nv-MSIs. Compared with traditional classification methods, CNN achieved better estimation accuracy and required minimal data preprocessing. Also, the specific region on Nv-MSIs and weight of each neurotransmitter that affected the classification most could be unraveled by CNN. Finally, the method was successfully applied to assist the identification of a new psychoactive substance seized recently. This sample was identified as cannabinoids, which greatly promoted the screening process.

Artificial Intelligence and Computational Biology in Gene Therapy: A Review

One of the trending fields in almost all areas of science and technology is artificial intelligence. Computational biology and artificial intelligence can help gene therapy in many steps including: gene identification, gene editing, vector design, development of new macromolecules and modeling of gene delivery. There are various tools used by computational biology and artificial intelligence in this field, such as genomics, transcriptomic and proteomics data analysis, machine learning algorithms and molecular interaction studies. These tools can introduce new gene targets, novel vectors, optimized experiment conditions, predict the outcomes and suggest the best solutions to avoid undesired immune responses following gene therapy treatment.

Mass spectrometry imaging methods for visualizing tumor heterogeneity

Profiling spatial distributions of lipids, metabolites, and proteins in tumors can reveal unique cellular microenvironments and provide molecular evidence for cancer cell dysfunction and proliferation. Mass spectrometry imaging (MSI) is a label-free technique that can be used to map biomolecules in tumors in situ. Here, we discuss current progress in applying MSI to uncover molecular heterogeneity in tumors. First, the analytical strategies to profile small molecules and proteins are outlined, and current methods for multimodal imaging to maximize biological information are highlighted. Second, we present and summarize biological insights obtained by MSI of tumor tissue. Finally, we discuss important considerations for designing MSI experiments and several current analytical challenges.

A Machine Learning-Driven Comparison of Ion Images Obtained by MALDI and MALDI-2 Mass Spectrometry Imaging

Matrix-assisted laser desorption ionization mass spectrometry imaging (MALDI-MSI) enables label-free imaging of biomolecules in biological tissues. However, many species remain undetected due to their poor ionization efficiencies. MALDI-2 (laser-induced post-ionization) is the most widely used post-ionization method for improving analyte ionization efficiencies. Mass spectra acquired using MALDI-2 constitute a combination of ions generated by both MALDI and MALDI-2 processes. Until now, no studies have focused on a detailed comparison between the ion images (as opposed to the generated m/z values) produced by MALDI and MALDI-2 for mass spectrometry imaging (MSI) experiments. Herein, we investigated the ion images produced by both MALDI and MALDI-2 on the same tissue section using correlation analysis (to explore similarities in ion images for ions common to both MALDI and MALDI-2) and a deep learning approach. For the latter, we used an analytical workflow based on the Xception convolutional neural network, which was originally trained for human-like natural image classification but which we adapted to elucidate similarities and differences in ion images obtained using the two MSI techniques. Correlation analysis demonstrated that common ions yielded similar spatial distributions with low-correlation species explained by either poor signal intensity in MALDI or the generation of additional unresolved signals using MALDI-2. Using the Xception-based method, we identified many regions in the t-SNE space of spatially similar ion images containing MALDI and MALDI-2-related signals. More notably, the method revealed distinct regions containing only MALDI-2 ion images with unique spatial distributions that were not observed using MALDI. These data explicitly demonstrate the ability of MALDI-2 to reveal molecular features and patterns as well as histological regions of interest that are not visible when using conventional MALDI.

Advances in mass spectrometry imaging for spatial cancer metabolomics

Mass spectrometry (MS) has become a central technique in cancer research. The ability to analyze various types of biomolecules in complex biological matrices makes it well suited for understanding biochemical alterations associated with disease progression. Different biological samples, including serum, urine, saliva, and tissues have been successfully analyzed using mass spectrometry. In particular, spatial metabolomics using MS imaging (MSI) allows the direct visualization of metabolite distributions in tissues, thus enabling in-depth understanding of cancer-associated biochemical changes within specific structures. In recent years, MSI studies have been increasingly used to uncover metabolic reprogramming associated with cancer development, enabling the discovery of key biomarkers with potential for cancer diagnostics. In this review, we aim to cover the basic principles of MSI experiments for the nonspecialists, including fundamentals, the sample preparation process, the evolution of the mass spectrometry techniques used, and data analysis strategies. We also review MSI advances associated with cancer research in the last 5 years, including spatial lipidomics and glycomics, the adoption of three-dimensional and multimodal imaging MSI approaches, and the implementation of artificial intelligence/machine learning in MSI-based cancer studies. The adoption of MSI in clinical research and for single-cell metabolomics is also discussed. Spatially resolved studies on other small molecule metabolites such as amino acids, polyamines, and nucleotides/nucleosides will not be discussed in the context.

The journey towards clinical adoption of MALDI-MS-based imaging proteomics: from current challenges to future expectations

Among the spatial omics techniques available, mass spectrometry imaging (MSI) represents one of the most promising owing to its capability to map the distribution of hundreds of peptides and proteins, as well as other classes of biomolecules, within a complex sample background in a multiplexed and relatively high-throughput manner. In particular, matrix-assisted laser desorption/ionisation (MALDI-MSI) has come to the fore and established itself as the most widely used technique in clinical research. However, the march of this technique towards clinical utility has been hindered by issues related to method reproducibility, appropriate biocomputational tools, and data storage. Notwithstanding these challenges, significant progress has been achieved in recent years regarding multiple facets of the technology and has rendered it more suitable for a possible clinical role. As such, there is now more robust and extensive evidence to suggest that the technology has the potential to support clinical decision-making processes under appropriate circumstances. In this review, we will discuss some of the recent developments that have facilitated this progress and outline some of the more promising clinical proteomics applications which have been developed with a clear goal towards implementation in mind.

rMSIfragment: improving MALDI-MSI lipidomics through automated in-source fragment annotation

Matrix-Assisted Laser Desorption Ionization Mass Spectrometry Imaging (MALDI-MSI) spatially resolves the chemical composition of tissues. Lipids are of particular interest, as they influence important biological processes in health and disease. However, the identification of lipids in MALDI-MSI remains a challenge due to the lack of chromatographic separation or untargeted tandem mass spectrometry. Recent studies have proposed the use of MALDI in-source fragmentation to infer structural information and aid identification. Here we present rMSIfragment, an open-source R package that exploits known adducts and fragmentation pathways to confidently annotate lipids in MALDI-MSI. The annotations are ranked using a novel score that demonstrates an area under the curve of 0.7 in ROC analyses using HPLC-MS and Target-Decoy validations. rMSIfragment applies to multiple MALDI-MSI sample types and experimental setups. Finally, we demonstrate that overlooking in-source fragments increases the number of incorrect annotations. Annotation workflows should consider in-source fragmentation tools such as rMSIfragment to increase annotation confidence and reduce the number of false positives.

Supervised topological data analysis for MALDI mass spectrometry imaging applications

BACKGROUND

Matrix-assisted laser desorption/ionization mass spectrometry imaging (MALDI MSI) displays significant potential for applications in cancer research, especially in tumor typing and subtyping. Lung cancer is the primary cause of tumor-related deaths, where the most lethal entities are adenocarcinoma (ADC) and squamous cell carcinoma (SqCC). Distinguishing between these two common subtypes is crucial for therapy decisions and successful patient management.

RESULTS

We propose a new algebraic topological framework, which obtains intrinsic information from MALDI data and transforms it to reflect topological persistence. Our framework offers two main advantages. Firstly, topological persistence aids in distinguishing the signal from noise. Secondly, it compresses the MALDI data, saving storage space and optimizes computational time for subsequent classification tasks. We present an algorithm that efficiently implements our topological framework, relying on a single tuning parameter. Afterwards, logistic regression and random forest classifiers are employed on the extracted persistence features, thereby accomplishing an automated tumor (sub-)typing process. To demonstrate the competitiveness of our proposed framework, we conduct experiments on a real-world MALDI dataset using cross-validation. Furthermore, we showcase the effectiveness of the single denoising parameter by evaluating its performance on synthetic MALDI images with varying levels of noise.

CONCLUSION

Our empirical experiments demonstrate that the proposed algebraic topological framework successfully captures and leverages the intrinsic spectral information from MALDI data, leading to competitive results in classifying lung cancer subtypes. Moreover, the framework's ability to be fine-tuned for denoising highlights its versatility and potential for enhancing data analysis in MALDI applications.

Multiple multi-sample testing under arbitrary covariance dependency

Modern high-throughput biomedical devices routinely produce data on a large scale, and the analysis of high-dimensional datasets has become commonplace in biomedical studies. However, given thousands or tens of thousands of measured variables in these datasets, extracting meaningful features poses a challenge. In this article, we propose a procedure to evaluate the strength of the associations between a nominal (categorical) response variable and multiple features simultaneously. Specifically, we propose a framework of large-scale multiple testing under arbitrary correlation dependency among test statistics. First, marginal multinomial regressions are performed for each feature individually. Second, we use an approach of multiple marginal models for each baseline-category pair to establish asymptotic joint normality of the stacked vector of the marginal multinomial regression coefficients. Third, we estimate the (limiting) covariance matrix between the estimated coefficients from all marginal models. Finally, our approach approximates the realized false discovery proportion of a thresholding procedure for the marginal p-values for each baseline-category logit pair. The proposed approach offers a sensible trade-off between the expected numbers of true and false findings. Furthermore, we demonstrate a practical application of the method on hyperspectral imaging data. This dataset is obtained by a matrix-assisted laser desorption/ionization (MALDI) instrument. MALDI demonstrates tremendous potential for clinical diagnosis, particularly for cancer research. In our application, the nominal response categories represent cancer (sub-)types.

XAI-enabled neural network analysis of metabolite spatial distributions

We used deep neural networks to process the mass spectrometry imaging (MSI) data of mouse muscle (young vs aged) and human cancer (tumor vs normal adjacent) tissues, with the aim of using explainable artificial intelligence (XAI) methods to rapidly identify biomarkers that can distinguish different classes of tissues, from several thousands of metabolite features. We also modified classic neural network architectures to construct a deep convolutional neural network that is more suitable for processing high-dimensional MSI data directly, instead of using dimension reduction techniques, and compared it to seven other machine learning analysis methods' performance in classification accuracy. After ascertaining the superiority of Channel-ResNet10, we used a novel channel selection-based XAI method to identify the key metabolite features that were responsible for its learning accuracy. These key metabolite biomarkers were then processed using MetaboAnalyst for pathway enrichment mapping. We found that Channel-ResNet10 was superior to seven other machine learning methods for MSI analysis, reaching > 98% accuracy in muscle aging and colorectal cancer datasets. We also used a novel channel selection-based XAI method to find that in young and aged muscle tissues, the differentially distributed metabolite biomarkers were especially enriched in the propanoate metabolism pathway, suggesting it as a novel target pathway for anti-aging therapy.

Multiple two-sample testing under arbitrary covariance dependency with an application in imaging mass spectrometry

Large-scale hypothesis testing has become a ubiquitous problem in high-dimensional statistical inference, with broad applications in various scientific disciplines. One relevant application is constituted by imaging mass spectrometry (IMS) association studies, where a large number of tests are performed simultaneously in order to identify molecular masses that are associated with a particular phenotype, for example, a cancer subtype. Mass spectra obtained from matrix-assisted laser desorption/ionization (MALDI) experiments are dependent, when considered as statistical quantities. False discovery proportion (FDP) estimation and  control under arbitrary dependency structure among test statistics is an active topic in modern multiple testing research. In this context, we are concerned with the evaluation of associations between the binary outcome variable (describing the phenotype) and multiple predictors derived from MALDI measurements. We propose an inference procedure in which the correlation matrix of the test statistics is utilized. The approach is based on multiple marginal models. Specifically, we fit a marginal logistic regression model for each predictor individually. Asymptotic joint normality of the stacked vector of the marginal regression coefficients is established under standard regularity assumptions, and their (limiting) correlation matrix is estimated. The proposed method extracts common factors from the resulting empirical correlation matrix. Finally, we estimate the realized FDP of a thresholding procedure for the marginal p-values. We demonstrate a practical application of the proposed workflow to MALDI IMS data in an oncological context.

Classification of Pancreatic Ductal Adenocarcinoma Using MALDI Mass Spectrometry Imaging Combined with Neural Networks

Despite numerous diagnostic and therapeutic advances, pancreatic ductal adenocarcinoma (PDAC) has a high mortality rate, and is the fourth leading cause of cancer death in developing countries. Besides its increasing prevalence, pancreatic malignancies are characterized by poor prognosis. Omics technologies have potential relevance for PDAC assessment but are time-intensive and relatively cost-intensive and limited by tissue heterogeneity. Matrix-assisted laser desorption/ionization mass spectrometry imaging (MALDI-MSI) can obtain spatially distinct peptide-signatures and enables tumor classification within a feasible time with relatively low cost. While MALDI-MSI data sets are inherently large, machine learning methods have the potential to greatly decrease processing time. We present a pilot study investigating the potential of MALDI-MSI in combination with neural networks, for classification of pancreatic ductal adenocarcinoma. Neural-network models were trained to distinguish between pancreatic ductal adenocarcinoma and other pancreatic cancer types. The proposed methods are able to correctly classify the PDAC types with an accuracy of up to 86% and a sensitivity of 82%. This study demonstrates that machine learning tools are able to identify different pancreatic carcinoma from complex MALDI data, enabling fast prediction of large data sets. Our results encourage a more frequent use of MALDI-MSI and machine learning in histopathological studies in the future.

Divide and Conquer: A Flexible Deep Learning Strategy for Exploring Metabolic Heterogeneity from Mass Spectrometry Imaging Data

Research on metabolic heterogeneity provides an important basis for the study of the molecular mechanism of a disease and personalized treatment. The screening of metabolism-related sub-regions that affect disease development is essential for the more focused exploration on disease progress aberrant phenotypes, even carcinogenesis and metastasis. The mass spectrometry imaging (MSI) technique has distinct advantages to reveal the heterogeneity of an organism based on in situ molecular profiles. The challenge of heterogeneous analysis has been to perform an objective identification among biological tissues with different characteristics. By introducing the divide-and-conquer strategy to architecture design and application, we establish here a flexible unsupervised deep learning model, called divide-and-conquer (dc)-DeepMSI, for metabolic heterogeneity analysis from MSI data without prior knowledge of histology. dc-DeepMSI can be used to identify either spatially contiguous regions of interest (ROIs) or spatially sporadic ROIs by designing two specific modes, spat-contig and spat-spor. Comparison results on fetus mouse data demonstrate that the dc-DeepMSI outperforms state-of-the-art MSI segmentation methods. We demonstrate that the novel learning strategy successfully obtained sub-regions that are statistically linked to the invasion status and molecular phenotypes of breast cancer as well as organizing principles during developmental phase.

Key defatting tissue pretreatment protocol for enhanced MALDI MS Imaging of peptide biomarkers visualization in the castor beans and their attribution applications

INTRODUCTION

Castor bean or ricin-induced intoxication or terror events have threatened public security and social safety. Potential resources or materials include beans, raw extraction products, crude toxins, and purified ricin. The traceability of the origins of castor beans is thus essential for forensic and anti-terror investigations. As a new imaging technique with label-free, rapid, and high throughput features, matrix-assisted laser desorption ionization mass spectrometry imaging (MALDI-MSI) has been gradually stressed in plant research. However, sample preparation approaches for plant tissues still face severe challenges, especially for some lipid-rich, water-rich, or fragile tissues. Proper tissue washing procedures would be pivotal, but little information is known until now.

METHODS

For castor beans containing plenty of lipids that were fragile when handled, we developed a comprehensive tissue pretreatment protocol. Eight washing procedures aimed at removing lipids were discussed in detail. We then constructed a robust MALDI-MSI method to enhance the detection sensitivity of RCBs in castor beans.

RESULTS AND DISCUSSION

A modified six-step washing procedure was chosen as the most critical parameter regarding the MSI visualization of peptides. The method was further applied to visualize and quantify the defense peptides, Ricinus communis biomarkers (RCBs) in castor bean tissue sections from nine different geographic sources from China, Pakistan, and Ethiopia. Multivariate statistical models, including deep learning network, revealed a valuable classification clue concerning nationality and altitude.

Multimodal Lung Cancer Subtyping Using Deep Learning Neural Networks on Whole Slide Tissue Images and MALDI MSI

Artificial intelligence (AI) has shown potential for facilitating the detection and classification of tumors. In patients with non-small cell lung cancer, distinguishing between the most common subtypes, adenocarcinoma (ADC) and squamous cell carcinoma (SqCC), is crucial for the development of an effective treatment plan. This task, however, may still present challenges in clinical routine. We propose a two-modality, AI-based classification algorithm to detect and subtype tumor areas, which combines information from matrix-assisted laser desorption/ionization (MALDI) mass spectrometry imaging (MSI) data and digital microscopy whole slide images (WSIs) of lung tissue sections. The method consists of first detecting areas with high tumor cell content by performing a segmentation of the hematoxylin and eosin-stained (H&E-stained) WSIs, and subsequently classifying the tumor areas based on the corresponding MALDI MSI data. We trained the algorithm on six tissue microarrays (TMAs) with tumor samples from N = 232 patients and used 14 additional whole sections for validation and model selection. Classification accuracy was evaluated on a test dataset with another 16 whole sections. The algorithm accurately detected and classified tumor areas, yielding a test accuracy of 94.7% on spectrum level, and correctly classified 15 of 16 test sections. When an additional quality control criterion was introduced, a 100% test accuracy was achieved on sections that passed the quality control (14 of 16). The presented method provides a step further towards the inclusion of AI and MALDI MSI data into clinical routine and has the potential to reduce the pathologist's work load. A careful analysis of the results revealed specific challenges to be considered when training neural networks on data from lung cancer tissue.

Emerging Computational Methods in Mass Spectrometry Imaging

Mass spectrometry imaging (MSI) is a powerful analytical technique that generates maps of hundreds of molecules in biological samples with high sensitivity and molecular specificity. Advanced MSI platforms with capability of high-spatial resolution and high-throughput acquisition generate vast amount of data, which necessitates the development of computational tools for MSI data analysis. In addition, computation-driven MSI experiments have recently emerged as enabling technologies for further improving the MSI capabilities with little or no hardware modification. This review provides a critical summary of computational methods and resources developed for MSI data analysis and interpretation along with computational approaches for improving throughput and molecular coverage in MSI experiments. This review is focused on the recently developed artificial intelligence methods and provides an outlook for a future paradigm shift in MSI with transformative computational methods.

Changes of Mass Spectra Patterns on a Brain Tissue Section Revealed by Deep Learning with Imaging Mass Spectrometry Data

The characteristic patterns of mass spectra in imaging mass spectrometry (IMS) strongly reflect the tissue environment. However, the boundaries formed where different tissue environments collide have not been visually assessed. In this study, IMS and convolutional neural network (CNN), one of the deep learning methods, were applied to the extraction of characteristic mass spectra patterns from training brain regions on rodents' brain sections. CNN produced classification models with high accuracy and low loss rate in any test data sets of mouse coronal sections measured by desorption electrospray ionization (DESI)-IMS and of mouse and rat sagittal sections by matrix-assisted laser desorption (MALDI)-IMS. On the basis of the extracted mass spectra pattern features, the histologically plausible segmentation and classification score imaging of the brain sections were obtained. The boundary imaging generated from classification scores showed the extreme changes of mass spectra patterns between the tissue environments, with no significant buffer zones for the intermediate state. The CNN-based analysis of IMS data is a useful tool for visually assessing the changes of mass spectra patterns on a tissue section, and it will contribute to a comprehensive view of the tissue environment.

Multi-Class Cancer Subtyping in Salivary Gland Carcinomas with MALDI Imaging and Deep Learning

Salivary gland carcinomas (SGC) are a heterogeneous group of tumors. The prognosis varies strongly according to its type, and even the distinction between benign and malign tumor is challenging. Adenoid cystic carcinoma (AdCy) is one subgroup of SGCs that is prone to late metastasis. This makes accurate tumor subtyping an important task. Matrix-assisted laser desorption/ionization (MALDI) imaging is a label-free technique capable of providing spatially resolved information about the abundance of biomolecules according to their mass-to-charge ratio. We analyzed tissue micro arrays (TMAs) of 25 patients (including six different SGC subtypes and a healthy control group of six patients) with high mass resolution MALDI imaging using a 12-Tesla magnetic resonance mass spectrometer. The high mass resolution allowed us to accurately detect single masses, with strong contributions to each class prediction. To address the added complexity created by the high mass resolution and multiple classes, we propose a deep-learning model. We showed that our deep-learning model provides a per-class classification accuracy of greater than 80% with little preprocessing. Based on this classification, we employed methods of explainable artificial intelligence (AI) to gain further insights into the spectrometric features of AdCys.

Deep learning-based pseudo-mass spectrometry imaging analysis for precision medicine

Liquid chromatography-mass spectrometry (LC-MS)-based untargeted metabolomics provides systematic profiling of metabolic. Yet, its applications in precision medicine (disease diagnosis) have been limited by several challenges, including metabolite identification, information loss and low reproducibility. Here, we present the deep-learning-based Pseudo-Mass Spectrometry Imaging (deepPseudoMSI) project (https://www.deeppseudomsi.org/), which converts LC-MS raw data to pseudo-MS images and then processes them by deep learning for precision medicine, such as disease diagnosis. Extensive tests based on real data demonstrated the superiority of deepPseudoMSI over traditional approaches and the capacity of our method to achieve an accurate individualized diagnosis. Our framework lays the foundation for future metabolic-based precision medicine.

Fully Automated Unconstrained Analysis of High-Resolution Mass Spectrometry Data with Machine Learning

Mass spectrometry (MS) is a convenient, highly sensitive, and reliable method for the analysis of complex mixtures, which is vital for materials science, life sciences fields such as metabolomics and proteomics, and mechanistic research in chemistry. Although it is one of the most powerful methods for individual compound detection, complete signal assignment in complex mixtures is still a great challenge. The unconstrained formula-generating algorithm, covering the entire spectra and revealing components, is a "dream tool" for researchers. We present the framework for efficient MS data interpretation, describing a novel approach for detailed analysis based on deisotoping performed by gradient-boosted decision trees and a neural network that generates molecular formulas from the fine isotopic structure, approaching the long-standing inverse spectral problem. The methods were successfully tested on three examples: fragment ion analysis in protein sequencing for proteomics, analysis of the natural samples for life sciences, and study of the cross-coupling catalytic system for chemistry.

Towards In-Situ Geochemical Analysis of Planetary Rocks and Soils by Laser Ablation/Ionisation Time-of-Flight Mass Spectrometry

Spectroscopic instruments were a part of payloads on orbiter and lander missions and delivered vast data sets to explore minerals, elements and molecules on air-less rocky planets, asteroids and comets on global and local scales. To answer current space science questions, the chemical composition of planetary rocks and soils at grain scale is required, as well as measurements of element (isotope) concentrations down to the part per million or lower. Only mass spectrometric methods equipped with laser sampling ion sources can deliver the necessary information. Laser sampling techniques can reduce the dimensions of the investigated sample material down to micrometre scale, allowing for the composition analysis of grain-sized objects or thin mineral layers with sufficiently high spatial resolution, such that important geological processes can be recognised and studied as they progressed in time. We describe the performance characteristics, when applied to meteorite and geological samples, of a miniaturised laser ablation/ionisation mass spectrometer (named LMS) system that has been developed in our group. The main advantages of the LMS instrument over competing techniques are illustrated by examples of high spatial (lateral and vertical) resolution studies in different meteorites, terrestrial minerals and fossil-like structures in ancient rocks for most elements of geochemical interest. Top-level parameters, such as dimension, weight, and power consumption of a possible flight design of the LMS system are presented as well.

AI/ML-driven advances in untargeted metabolomics and exposomics for biomedical applications

Metabolomics describes a high-throughput approach for measuring a repertoire of metabolites and small molecules in biological samples. One utility of untargeted metabolomics, unbiased global analysis of the metabolome, is to detect key metabolites as contributors to, or readouts of, human health and disease. In this perspective, we discuss how artificial intelligence (AI) and machine learning (ML) have promoted major advances in untargeted metabolomics workflows and facilitated pivotal findings in the areas of disease screening and diagnosis. We contextualize applications of AI and ML to the emerging field of high-resolution mass spectrometry (HRMS) exposomics, which unbiasedly detects endogenous metabolites and exogenous chemicals in human tissue to characterize exposure linked with disease outcomes. We discuss the state of the science and suggest potential opportunities for using AI and ML to improve data quality, rigor, detection, and chemical identification in untargeted metabolomics and exposomics studies.

What are we imaging? Software tools and experimental strategies for annotation and identification of small molecules in mass spectrometry imaging

Mass spectrometry imaging (MSI) has become a widespread analytical technique to perform nonlabeled spatial molecular identification. The Achilles' heel of MSI is the annotation and identification of molecular species due to intrinsic limitations of the technique (lack of chromatographic separation and the difficulty to apply tandem MS). Successful strategies to perform annotation and identification combine extra analytical steps, like using orthogonal analytical techniques to identify compounds; with algorithms that integrate the spectral and spatial information. In this review, we discuss different experimental strategies and bioinformatics tools to annotate and identify compounds in MSI experiments. We target strategies and tools for small molecule applications, such as lipidomics and metabolomics. First, we explain how sample preparation and the acquisition process influences annotation and identification, from sample preservation to the use of orthogonal techniques. Then, we review twelve software tools for annotation and identification in MSI. Finally, we offer perspectives on two current needs of the MSI community: the adaptation of guidelines for communicating confidence levels in identifications; and the creation of a standard format to store and exchange annotations and identifications in MSI.

Robust subtyping of non-small cell lung cancer whole sections through MALDI mass spectrometry imaging

Subtyping of the most common non-small cell lung cancer (NSCLC) tumor types adenocarcinoma (ADC) and squamous cell carcinoma (SqCC) is still a challenge in the clinical routine and a correct diagnosis is crucial for an adequate therapy selection. Matrix-assisted laser desorption/ionization (MALDI) mass spectrometry imaging (MSI) has shown potential for NSCLC subtyping but is subject to strong technical variability and has only been applied to tissue samples assembled in tissue microarrays (TMAs). To our knowledge, a successful transfer of a classifier from TMAs to whole sections, which are generated in the standard clinical routine, has not been presented in the literature as of yet. We introduce a classification algorithm using extensive preprocessing and a classifier (either a neural network or a linear discriminant analysis (LDA)) to robustly classify whole sections of ADC and SqCC lung tissue. The classifiers were trained on TMAs and validated and tested on whole sections. Vital for a successful application on whole sections is the extensive preprocessing and the use of whole sections for hyperparameter selection. The classification system with the neural network/LDA results in 99.0%/98.3% test accuracy on spectra level and 100.0%/100.0% test accuracy on whole section level, respectively, and, therefore, provides a powerful tool to support the pathologist's decision making process. The presented method is a step further towards a clinical application of MALDI MSI and artificial intelligence for subtyping of NSCLC tissue sections.

massNet: integrated processing and classification of spatially resolved mass spectrometry data using deep learning for rapid tumor delineation

MOTIVATION

Mass spectrometry imaging (MSI) provides rich biochemical information in a label-free manner and therefore holds promise to substantially impact current practice in disease diagnosis. However, the complex nature of MSI data poses computational challenges in its analysis. The complexity of the data arises from its large size, high-dimensionality and spectral nonlinearity. Preprocessing, including peak picking, has been used to reduce raw data complexity; however, peak picking is sensitive to parameter selection that, perhaps prematurely, shapes the downstream analysis for tissue classification and ensuing biological interpretation.

RESULTS

We propose a deep learning model, massNet, that provides the desired qualities of scalability, nonlinearity and speed in MSI data analysis. This deep learning model was used, without prior preprocessing and peak picking, to classify MSI data from a mouse brain harboring a patient-derived tumor. The massNet architecture established automatically learning of predictive features, and automated methods were incorporated to identify peaks with potential for tumor delineation. The model's performance was assessed using cross-validation, and the results demonstrate higher accuracy and a substantial gain in speed compared to the established classical machine learning method, support vector machine.

AVAILABILITY AND IMPLEMENTATION

https://github.com/wabdelmoula/massNet. The data underlying this article are available in the NIH Common Fund's National Metabolomics Data Repository (NMDR) Metabolomics Workbench under project id (PR001292) with http://dx.doi.org/10.21228/M8Q70T.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Design of Optimal Deep Learning-Based Pancreatic Tumor and Nontumor Classification Model Using Computed Tomography Scans

Pancreatic tumor is a lethal kind of tumor and its prediction is really poor in the current scenario. Automated pancreatic tumor classification using computer-aided diagnosis (CAD) model is necessary to track, predict, and classify the existence of pancreatic tumors. Artificial intelligence (AI) can offer extensive diagnostic expertise and accurate interventional image interpretation. With this motivation, this study designs an optimal deep learning based pancreatic tumor and nontumor classification (ODL-PTNTC) model using CT images. The goal of the ODL-PTNTC technique is to detect and classify the existence of pancreatic tumors and nontumor. The proposed ODL-PTNTC technique includes adaptive window filtering (AWF) technique to remove noise existing in it. In addition, sailfish optimizer based Kapur's Thresholding (SFO-KT) technique is employed for image segmentation process. Moreover, feature extraction using Capsule Network (CapsNet) is derived to generate a set of feature vectors. Furthermore, Political Optimizer (PO) with Cascade Forward Neural Network (CFNN) is employed for classification purposes. In order to validate the enhanced performance of the ODL-PTNTC technique, a series of simulations take place and the results are investigated under several aspects. A comprehensive comparative results analysis stated the promising performance of the ODL-PTNTC technique over the recent approaches.

MS2AI: automated repurposing of public peptide LC-MS data for machine learning applications

MOTIVATION

Liquid-chromatography mass-spectrometry (LC-MS) is the established standard for analyzing the proteome in biological samples by identification and quantification of thousands of proteins. Machine learning (ML) promises to considerably improve the analysis of the resulting data, however, there is yet to be any tool that mediates the path from raw data to modern ML applications. More specifically, ML applications are currently hampered by three major limitations: (i) absence of balanced training data with large sample size; (ii) unclear definition of sufficiently information-rich data representations for e.g. peptide identification; (iii) lack of benchmarking of ML methods on specific LC-MS problems.

RESULTS

We created the MS2AI pipeline that automates the process of gathering vast quantities of MS data for large-scale ML applications. The software retrieves raw data from either in-house sources or from the proteomics identifications database, PRIDE. Subsequently, the raw data are stored in a standardized format amenable for ML, encompassing MS1/MS2 spectra and peptide identifications. This tool bridges the gap between MS and AI, and to this effect we also present an ML application in the form of a convolutional neural network for the identification of oxidized peptides.

AVAILABILITY AND IMPLEMENTATION

An open-source implementation of the software can be found at https://gitlab.com/roettgerlab/ms2ai.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Application of Intraoperative Mass Spectrometry and Data Analytics for Oncological Margin Detection, A Review

OBJECTIVE

A common phase of early-stage oncological treatment is the surgical resection of cancerous tissue. The presence of cancer cells on the resection margin, referred to as positive margin, is correlated with the recurrence of cancer and may require re-operation, negatively impacting many facets of patient outcomes. There exists a significant gap in the surgeons ability to intraoperatively delineate between tissues. Mass spectrometry methods have shown considerable promise as intraoperative tissue profiling tools that can assist with the complete resection of cancer. To do so, the vastness of the information collected through these modalities must be digested, relying on robust and efficient extraction of insights through data analysis pipelines.

METHODS

We review clinical mass spectrometry literature and prioritize intraoperatively applied modalities. We also survey the data analysis methods employed in these studies.

RESULTS

Our review outlines the advantages and shortcomings of mass spectrometry imaging and point-based tissue probing methods. For each modality, we identify statistical, linear transformation and machine learning techniques that demonstrate high performance in classifying cancerous tissues across several organ systems. A limited number of studies presented results captured intraoperatively.

CONCLUSION

Through continued research of data centric techniques, like mass spectrometry, and the development of robust analysis approaches, intraoperative margin assessment is becoming feasible.

SIGNIFICANCE

By establishing the relatively short history of mass spectrometry techniques applied to surgical studies, we hope to inform future applications and aid in the selection of suitable data analysis frameworks for the development of intraoperative margin detection technologies.

High-Throughput Molecular Imaging via Deep-Learning-Enabled Raman Spectroscopy

Raman spectroscopy enables nondestructive, label-free imaging with unprecedented molecular contrast, but is limited by slow data acquisition, largely preventing high-throughput imaging applications. Here, we present a comprehensive framework for higher-throughput molecular imaging via deep-learning-enabled Raman spectroscopy, termed DeepeR, trained on a large data set of hyperspectral Raman images, with over 1.5 million spectra (400 h of acquisition) in total. We first perform denoising and reconstruction of low signal-to-noise ratio Raman molecular signatures via deep learning, with a 10× improvement in the mean-squared error over common Raman filtering methods. Next, we develop a neural network for robust 2-4× spatial super-resolution of hyperspectral Raman images that preserve molecular cellular information. Combining these approaches, we achieve Raman imaging speed-ups of up to 40-90×, enabling good-quality cellular imaging with a high-resolution, high signal-to-noise ratio in under 1 min. We further demonstrate Raman imaging speed-up of 160×, useful for lower resolution imaging applications such as the rapid screening of large areas or for spectral pathology. Finally, transfer learning is applied to extend DeepeR from cell to tissue-scale imaging. DeepeR provides a foundation that will enable a host of higher-throughput Raman spectroscopy and molecular imaging applications across biomedicine.

A Comprehensive Survey on Deep-Learning-Based Breast Cancer Diagnosis

Breast cancer is now the most frequently diagnosed cancer in women, and its percentage is gradually increasing. Optimistically, there is a good chance of recovery from breast cancer if identified and treated at an early stage. Therefore, several researchers have established deep-learning-based automated methods for their efficiency and accuracy in predicting the growth of cancer cells utilizing medical imaging modalities. As of yet, few review studies on breast cancer diagnosis are available that summarize some existing studies. However, these studies were unable to address emerging architectures and modalities in breast cancer diagnosis. This review focuses on the evolving architectures of deep learning for breast cancer detection. In what follows, this survey presents existing deep-learning-based architectures, analyzes the strengths and limitations of the existing studies, examines the used datasets, and reviews image pre-processing techniques. Furthermore, a concrete review of diverse imaging modalities, performance metrics and results, challenges, and research directions for future researchers is presented.

Machine learning meets omics: applications and perspectives

The innovation of biotechnologies has allowed the accumulation of omics data at an alarming rate, thus introducing the era of 'big data'. Extracting inherent valuable knowledge from various omics data remains a daunting problem in bioinformatics. Better solutions often need some kind of more innovative methods for efficient handlings and effective results. Recent advancements in integrated analysis and computational modeling of multi-omics data helped address such needs in an increasingly harmonious manner. The development and application of machine learning have largely advanced our insights into biology and biomedicine and greatly promoted the development of therapeutic strategies, especially for precision medicine. Here, we propose a comprehensive survey and discussion on what happened, is happening and will happen when machine learning meets omics. Specifically, we describe how artificial intelligence can be applied to omics studies and review recent advancements at the interface between machine learning and the ever-widest range of omics including genomics, transcriptomics, proteomics, metabolomics, radiomics, as well as those at the single-cell resolution. We also discuss and provide a synthesis of ideas, new insights, current challenges and perspectives of machine learning in omics.

Combining Image Features and Patient Metadata to Enhance Transfer Learning

In this work, we compare the performance of six state-of-the-art deep neural networks in classification tasks when using only image features, to when these are combined with patient metadata. We utilise transfer learning from networks pretrained on ImageNet to extract image features from the ISIC HAM10000 dataset prior to classification. Using several classification performance metrics, we evaluate the effects of including metadata with the image features. Furthermore, we repeat our experiments with data augmentation. Our results show an overall enhancement in performance of each network as assessed by all metrics, only noting degradation in a vgg16 architecture. Our results indicate that this performance enhancement may be a general property of deep networks and should be explored in other areas. Moreover, these improvements come at a negligible additional cost in computation time, and therefore are a practical method for other applications.

Cancer Tissue Classification Using Supervised Machine Learning Applied to MALDI Mass Spectrometry Imaging

Simple Summary

Classic histopathological examination of tissues remains the mainstay for cancer diagnosis and staging. However, in some cases histopathologic analysis yields ambiguous results, leading to inconclusive disease classification. We set out to explore the diagnostic potential of mass spectrometry-based imaging for tumour classification based on proteomic fingerprints. Combining mass spectrometry with supervised machine learning, we were able to distinguish colorectal tumor from normal tissue with an overall accuracy of 98%. In addition, this approach was able to predict the presence of lymph node metastasis in primary tumour of endometrial cancer with an overall accuracy of 80%. These results highlight the potential of this technology to determine the optimal treatment for cancer patients to reduce morbidity and improve patients’ outcomes.

Abstract

Matrix assisted laser desorption/ionization mass spectrometry imaging (MALDI MSI) can determine the spatial distribution of analytes such as protein distributions in a tissue section according to their mass-to-charge ratio. Here, we explored the clinical potential of machine learning (ML) applied to MALDI MSI data for cancer diagnostic classification using tissue microarrays (TMAs) on 302 colorectal (CRC) and 257 endometrial cancer (EC)) patients. ML based on deep neural networks discriminated colorectal tumour from normal tissue with an overall accuracy of 98% in balanced cross-validation (98.2% sensitivity and 98.6% specificity). Moreover, our machine learning approach predicted the presence of lymph node metastasis (LNM) for primary tumours of EC with an accuracy of 80% (90% sensitivity and 69% specificity). Our results demonstrate the capability of MALDI MSI for complementing classic histopathological examination for cancer diagnostic applications.

Peptide Signatures for Prognostic Markers of Pancreatic Cancer by MALDI Mass Spectrometry Imaging

Despite the overall poor prognosis of pancreatic cancer there is heterogeneity in clinical courses of tumors not assessed by conventional risk stratification. This yields the need of additional markers for proper assessment of prognosis and multimodal clinical management. We provide a proof of concept study evaluating the feasibility of Matrix-assisted laser desorption/ionization (MALDI) mass spectrometry imaging (MSI) to identify specific peptide signatures linked to prognostic parameters of pancreatic cancer. On 18 patients with exocrine pancreatic cancer after tumor resection, MALDI imaging analysis was performed additional to histopathological assessment. Principal component analysis (PCA) was used to explore discrimination of peptide signatures of prognostic histopathological features and receiver operator characteristic (ROC) to identify which specific m/z values are the most discriminative between the prognostic subgroups of patients. Out of 557 aligned m/z values discriminate peptide signatures for the prognostic histopathological features lymphatic vessel invasion (pL, 16 m/z values, eight proteins), nodal metastasis (pN, two m/z values, one protein) and angioinvasion (pV, 4 m/z values, two proteins) were identified. These results yield proof of concept that MALDI-MSI of pancreatic cancer tissue is feasible to identify peptide signatures of prognostic relevance and can augment risk assessment.

Peak learning of mass spectrometry imaging data using artificial neural networks

Mass spectrometry imaging (MSI) is an emerging technology that holds potential for improving, biomarker discovery, metabolomics research, pharmaceutical applications and clinical diagnosis. Despite many solutions being developed, the large data size and high dimensional nature of MSI, especially 3D datasets, still pose computational and memory complexities that hinder accurate identification of biologically relevant molecular patterns. Moreover, the subjectivity in the selection of parameters for conventional pre-processing approaches can lead to bias. Therefore, we assess if a probabilistic generative model based on a fully connected variational autoencoder can be used for unsupervised analysis and peak learning of MSI data to uncover hidden structures. The resulting msiPL method learns and visualizes the underlying non-linear spectral manifold, revealing biologically relevant clusters of tissue anatomy in a mouse kidney and tumor heterogeneity in human prostatectomy tissue, colorectal carcinoma, and glioblastoma mouse model, with identification of underlying m/z peaks. The method is applied for the analysis of MSI datasets ranging from 3.3 to 78.9 GB, without prior pre-processing and peak picking, and acquired using different mass spectrometers at different centers.

Automated biomarker candidate discovery in imaging mass spectrometry data through spatially localized Shapley additive explanations

The search for molecular species that are differentially expressed between biological states is an important step towards discovering promising biomarker candidates. In imaging mass spectrometry (IMS), performing this search manually is often impractical due to the large size and high-dimensionality of IMS datasets. Instead, we propose an interpretable machine learning workflow that automatically identifies biomarker candidates by their mass-to-charge ratios, and that quantitatively estimates their relevance to recognizing a given biological class using Shapley additive explanations (SHAP). The task of biomarker candidate discovery is translated into a feature ranking problem: given a classification model that assigns pixels to different biological classes on the basis of their mass spectra, the molecular species that the model uses as features are ranked in descending order of relative predictive importance such that the top-ranking features have a higher likelihood of being useful biomarkers. Besides providing the user with an experiment-wide measure of a molecular species' biomarker potential, our workflow delivers spatially localized explanations of the classification model's decision-making process in the form of a novel representation called SHAP maps. SHAP maps deliver insight into the spatial specificity of biomarker candidates by highlighting in which regions of the tissue sample each feature provides discriminative information and in which regions it does not. SHAP maps also enable one to determine whether the relationship between a biomarker candidate and a biological state of interest is correlative or anticorrelative. Our automated approach to estimating a molecular species' potential for characterizing a user-provided biological class, combined with the untargeted and multiplexed nature of IMS, allows for the rapid screening of thousands of molecular species and the obtention of a broader biomarker candidate shortlist than would be possible through targeted manual assessment. Our biomarker candidate discovery workflow is demonstrated on mouse-pup and rat kidney case studies.

Applying Probe Electrospray Ionization Mass Spectrometry to Cytological Diagnosis: A Preliminary Study by Using Cultured Lung Cancer Cells

OBJECTIVES

Cytology and histology are 2 indispensable diagnostic tools for cancer diagnosis, which are rapidly increasing in importance with aging populations. We applied mass spectrometry (MS) as a rapid approach for swiftly acquiring nonmorphological information of interested cells. Conventional MS, which primarily rely on promoting ionization by pre-applying a matrix to cells, has the drawback of time-consuming both on data acquisition and analysis. As an emerging method, probe electrospray ionization-MS (PESI-MS) with a dedicated probe is capable to pierce sample and measure specimen in small amounts, either liquid or solid, without the requirement for sample pretreatment. Furthermore, PESI-MS is timesaving compared to the conventional MS. Herein, we investigated the capability of PESI-MS to characterize the cell types derived from the respiratory tract of human tissues.

STUDY DESIGN

PESI-MS analyses with DPiMS-2020 were performed on various type of cultured cells including 5 lung squamous cell carcinomas, 5 lung adenocarcinomas, 5 small-cell carcinomas, 4 malignant mesotheliomas, and 2 normal controls.

RESULTS

Several characteristic peaks were detected at around m/z 200 and 800 that were common in all samples. As expected, partial least squares-discriminant analysis of PESI-MS data distinguished the cancer cell types from normal control cells. Moreover, distinct clusters divided squamous cell carcinoma from adenocarcinoma.

CONCLUSION

PESI-MS presented a promising potential as a novel diagnostic modality for swiftly acquiring specific cytological information.

Mass Spectrometry Imaging of Fibroblasts: Promise and Challenge

INTRODUCTION

Fibroblasts maintain tissue and organ homeostasis through output of extracellular matrix that affects nearby cell signaling within the stroma. Altered fibroblast signaling contributes to many disease states and extracellular matrix secreted by fibroblasts has been used to stratify patient by outcome, recurrence, and therapeutic resistance. Recent advances in imaging mass spectrometry allow access to single cell fibroblasts and their ECM niche within clinically relevant tissue samples.

AREAS COVERED

We review biological and technical challenges as well as new solutions to proteomic access of fibroblast expression within the complex tissue microenvironment. Review topics cover conventional proteomic methods for single fibroblast analysis and current approaches to accessing single fibroblast proteomes by imaging mass spectrometry approaches. Strategies to target and evaluate the single cell stroma proteome on the basis of cell signaling are presented.

EXPERT OPINION

The promise of defining proteomic signatures from fibroblasts and their extracellular matrix niches is the discovery of new disease markers and the ability to refine therapeutic treatments. Several imaging mass spectrometry approaches exist to define the fibroblast in the setting of pathological changes from clinically acquired samples. Continued technology advances are needed to access and understand the stromal proteome and apply testing to the clinic.

Deeply Supervised UNet for Semantic Segmentation to Assist Dermatopathological Assessment of Basal Cell Carcinoma

Accurate and fast assessment of resection margins is an essential part of a dermatopathologist's clinical routine. In this work, we successfully develop a deep learning method to assist the dermatopathologists by marking critical regions that have a high probability of exhibiting pathological features in whole slide images (WSI). We focus on detecting basal cell carcinoma (BCC) through semantic segmentation using several models based on the UNet architecture. The study includes 650 WSI with 3443 tissue sections in total. Two clinical dermatopathologists annotated the data, marking tumor tissues' exact location on 100 WSI. The rest of the data, with ground-truth sectionwise labels, are used to further validate and test the models. We analyze two different encoders for the first part of the UNet network and two additional training strategies: (a) deep supervision, (b) linear combination of decoder outputs, and obtain some interpretations about what the network's decoder does in each case. The best model achieves over 96%, accuracy, sensitivity, and specificity on the Test set.

Spatially aware clustering of ion images in mass spectrometry imaging data using deep learning

Computational analysis is crucial to capitalize on the wealth of spatio-molecular information generated by mass spectrometry imaging (MSI) experiments. Currently, the spatial information available in MSI data is often under-utilized, due to the challenges of in-depth spatial pattern extraction. The advent of deep learning has greatly facilitated such complex spatial analysis. In this work, we use a pre-trained neural network to extract high-level features from ion images in MSI data, and test whether this improves downstream data analysis. The resulting neural network interpretation of ion images, coined neural ion images, is used to cluster ion images based on spatial expressions. We evaluate the impact of neural ion images on two ion image clustering pipelines, namely DBSCAN clustering, combined with UMAP-based dimensionality reduction, and k-means clustering. In both pipelines, we compare regular and neural ion images from two different MSI datasets. All tested pipelines could extract underlying spatial patterns, but the neural network-based pipelines provided better assignment of ion images, with more fine-grained clusters, and greater consistency in the spatial structures assigned to individual clusters. Additionally, we introduce the relative isotope ratio metric to quantitatively evaluate clustering quality. The resulting scores show that isotopical m/z values are more often clustered together in the neural network-based pipeline, indicating improved clustering outcomes. The usefulness of neural ion images extends beyond clustering towards a generic framework to incorporate spatial information into any MSI-focused machine learning pipeline, both supervised and unsupervised.

A versatile deep learning architecture for classification and label-free prediction of hyperspectral images

Hyperspectral imaging is a technique that provides rich chemical or compositional information not regularly available to traditional imaging modalities such as intensity imaging or color imaging based on the reflection, transmission, or emission of light. Analysis of hyperspectral imaging often relies on machine learning methods to extract information. Here, we present a new flexible architecture, the U-within-U-Net, that can perform classification, segmentation, and prediction of orthogonal imaging modalities on a variety of hyperspectral imaging techniques. Specifically, we demonstrate feature segmentation and classification on the Indian Pines hyperspectral dataset and simultaneous location prediction of multiple drugs in mass spectrometry imaging of rat liver tissue. We further demonstrate label-free fluorescence image prediction from hyperspectral stimulated Raman scattering microscopy images. The applicability of the U-within-U-Net architecture on diverse datasets with widely varying input and output dimensions and data sources suggest that it has great potential in advancing the use of hyperspectral imaging across many different application areas ranging from remote sensing, to medical imaging, to microscopy.

Deep multiple instance learning classifies subtissue locations in mass spectrometry images from tissue-level annotations

MOTIVATION

Mass spectrometry imaging (MSI) characterizes the molecular composition of tissues at spatial resolution, and has a strong potential for distinguishing tissue types, or disease states. This can be achieved by supervised classification, which takes as input MSI spectra, and assigns class labels to subtissue locations. Unfortunately, developing such classifiers is hindered by the limited availability of training sets with subtissue labels as the ground truth. Subtissue labeling is prohibitively expensive, and only rough annotations of the entire tissues are typically available. Classifiers trained on data with approximate labels have sub-optimal performance.

RESULTS

To alleviate this challenge, we contribute a semi-supervised approach mi-CNN. mi-CNN implements multiple instance learning with a convolutional neural network (CNN). The multiple instance aspect enables weak supervision from tissue-level annotations when classifying subtissue locations. The convolutional architecture of the CNN captures contextual dependencies between the spectral features. Evaluations on simulated and experimental datasets demonstrated that mi-CNN improved the subtissue classification as compared to traditional classifiers. We propose mi-CNN as an important step toward accurate subtissue classification in MSI, enabling rapid distinction between tissue types and disease states.

AVAILABILITY AND IMPLEMENTATION

The data and code are available at https://github.com/Vitek-Lab/mi-CNN_MSI.

Imaging Beyond Seeing: Early Prognosis of Cancer Treatment

Assessing cancer response to therapeutic interventions has been realized as an important course to early predict curative efficacy and treatment outcomes due to tumor heterogeneity. Compared to the traditional invasive tissue biopsy method, molecular imaging techniques have fundamentally revolutionized the ability to evaluate cancer response in a spatiotemporal manner. The past few years has witnessed a paradigm shift on the efforts from manufacturing functional molecular imaging probes for seeing a tumor to a vantage stage of interpreting the tumor response during different treatments. This review is to stand by the current development of advanced imaging technologies aiming to predict the treatment response in cancer therapy. Special interest is placed on the systems that are able to provide rapid and noninvasive assessment of pharmacokinetic drug fates (e.g., drug distribution, release, and activation) and tumor microenvironment heterogeneity (e.g., tumor cells, macrophages, dendritic cells (DCs), T cells, and inflammatory cells). The current status, practical significance, and future challenges of the emerging artificial intelligence (AI) technology and machine learning in the applications of medical imaging fields is overviewed. Ultimately, the authors hope that this review is timely to spur research interest in molecular imaging and precision medicine.

Feature selection and classification of noisy proteomics mass spectrometry data based on one-bit perturbed compressed sensing

MOTIVATION

The classification of high-throughput protein data based on mass spectrometry (MS) is of great practical significance in medical diagnosis. Generally, MS data are characterized by high dimension, which inevitably leads to prohibitive cost of computation. To solve this problem, one-bit compressed sensing (CS), which is an extreme case of quantized CS, has been employed on MS data to select important features with low dimension. Though enjoying remarkably reduction of computation complexity, the current one-bit CS method does not consider the unavoidable noise contained in MS dataset, and does not exploit the inherent structure of the underlying MS data.

RESULTS

We propose two feature selection (FS) methods based on one-bit CS to deal with the noise and the underlying block-sparsity features, respectively. In the first method, the FS problem is modeled as a perturbed one-bit CS problem, where the perturbation represents the noise in MS data. By iterating between perturbation refinement and FS, this method selects the significant features from noisy data. The second method formulates the problem as a perturbed one-bit block CS problem and selects the features block by block. Such block extraction is due to the fact that the significant features in the first method usually cluster in groups. Experiments show that, the two proposed methods have better classification performance for real MS data when compared with the existing method, and the second one outperforms the first one.

AVAILABILITY AND IMPLEMENTATION

The source code of our methods is available at: https://github.com/tianyan8023/OBCS.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Pneumatically Sprayed Gold Nanoparticles for Mass Spectrometry Imaging of Neurotransmitters

Using citrate-capped gold nanoparticles (AuNPs) for laser desorption ionization mass spectrometry (LDI-MS) is an approach that has demonstrated broad applicability to ionization of different classes of molecules. Here, we show a simple AuNP-based approach for the ionization of neurotransmitters. Specifically, the detection of acetylcholine, dopamine, epinephrine, glutamine, 4-aminobutyric acid, norepinephrine, octopamine, and serotonin was achieved at physiologically relevant concentrations in serum and homogenized tissue. Additionally, pneumatic spraying of AuNPs onto tissue sections facilitated mass spectrometry imaging (MSI) of rabbit brain tissue sections, zebrafish embryos, and neuroblastoma cells for several neurotransmitters simultaneously using this quick and simple sample preparation. AuNP LDI-MS achieved mapping of neurotransmitters in fine structures of zebrafish embryos and neuroblastoma cells at a lateral spatial resolution of 5 μm. The use of AuNPs to ionize small aminergic neurotransmitters in situ provides a fast, high-spatial resolution method for simultaneous detection of a class of molecules that typically evade comprehensive detection with traditional matrixes.

Cumulative learning enables convolutional neural network representations for small mass spectrometry data classification

Rapid and accurate clinical diagnosis remains challenging. A component of diagnosis tool development is the design of effective classification models with Mass spectrometry (MS) data. Some Machine Learning approaches have been investigated but these models require time-consuming preprocessing steps to remove artifacts, making them unsuitable for rapid analysis. Convolutional Neural Networks (CNNs) have been found to perform well under such circumstances since they can learn representations from raw data. However, their effectiveness decreases when the number of available training samples is small, which is a common situation in medicine. In this work, we investigate transfer learning on 1D-CNNs, then we develop a cumulative learning method when transfer learning is not powerful enough. We propose to train the same model through several classification tasks over various small datasets to accumulate knowledge in the resulting representation. By using rat brain as the initial training dataset, a cumulative learning approach can have a classification accuracy exceeding 98% for 1D clinical MS-data. We show the use of cumulative learning using datasets generated in different biological contexts, on different organisms, and acquired by different instruments. Here we show a promising strategy for improving MS data classification accuracy when only small numbers of samples are available.

Convolutional Neural Networks are powerful tools for clinical diagnosis but their effectiveness decreases when the number of available samples is small. Here, the authors develop a cumulative learning method by training the same model through several classification tasks over various small Mass Spectrometry datasets.

Histo-molecular differentiation of renal cancer subtypes by mass spectrometry imaging and rapid proteome profiling of formalin-fixed paraffin-embedded tumor tissue sections

Pathology differentiation of renal cancer types is challenging due to tissue similarities or overlapping histological features of various tumor (sub) types. As assessment is often manually conducted outcomes can be prone to human error and therefore require high-level expertise and experience. Mass spectrometry can provide detailed histo-molecular information on tissue and is becoming increasingly popular in clinical settings. Spatially resolving technologies such as mass spectrometry imaging and quantitative microproteomics profiling in combination with machine learning approaches provide promising tools for automated tumor classification of clinical tissue sections.

In this proof of concept study we used MALDI-MS imaging (MSI) and rapid LC-MS/MS-based microproteomics technologies (15 min/sample) to analyze formalin-fixed paraffin embedded (FFPE) tissue sections and classify renal oncocytoma (RO, n = 11), clear cell renal cell carcinoma (ccRCC, n = 12) and chromophobe renal cell carcinoma (ChRCC, n = 5). Both methods were able to distinguish ccRCC, RO and ChRCC in cross-validation experiments. MSI correctly classified 87% of the patients whereas the rapid LC-MS/MS-based microproteomics approach correctly classified 100% of the patients.

This strategy involving MSI and rapid proteome profiling by LC-MS/MS reveals molecular features of tumor sections and enables cancer subtype classification. Mass spectrometry provides a promising complementary approach to current pathological technologies for precise digitized diagnosis of diseases.