High definition infrared spectroscopic imaging for lymph node histopathology

Computational methods for metastasis detection in lymph nodes and characterization of the metastasis-free lymph node microarchitecture: A systematic-narrative hybrid review

Background

Histological examination of tumor draining lymph nodes (LNs) plays a vital role in cancer staging and prognostication. However, as soon as a LN is classed as metastasis-free, no further investigation will be performed and thus, potentially clinically relevant information detectable in tumor-free LNs is currently not captured.

Objective

To systematically study and critically assess methods for the analysis of digitized histological LN images described in published research.

Methods

A systematic search was conducted in several public databases up to December 2023 using relevant search terms. Studies using brightfield light microscopy images of hematoxylin and eosin or immunohistochemically stained LN tissue sections aiming to detect and/or segment LNs, their compartments or metastatic tumor using artificial intelligence (AI) were included. Dataset, AI methodology, cancer type, and study objective were compared between articles.

Results

A total of 7201 articles were collected and 73 articles remained for detailed analyses after article screening. Of the remaining articles, 86% aimed at LN metastasis identification, 8% aimed at LN compartment segmentation, and remaining focused on LN contouring. Furthermore, 78% of articles used patch classification and 22% used pixel segmentation models for analyses. Five out of six studies (83%) of metastasis-free LNs were performed on publicly unavailable datasets, making quantitative article comparison impossible.

Conclusions

Multi-scale models mimicking multiple microscopy zooms show promise for computational LN analysis. Large-scale datasets are needed to establish the clinical relevance of analyzing metastasis-free LN in detail. Further research is needed to identify clinically interpretable metrics for LN compartment characterization.

An optical photothermal infrared investigation of lymph nodal metastases of oral squamous cell carcinoma

In this study, optical photothermal infrared (O-PTIR) spectroscopy combined with machine learning algorithms were used to evaluate 46 tissue cores of surgically resected cervical lymph nodes, some of which harboured oral squamous cell carcinoma nodal metastasis. The ratios obtained between O-PTIR chemical images at 1252 cm-1 and 1285 cm-1 were able to reveal morphological details from tissue samples that are comparable to the information achieved by a pathologist's interpretation of optical microscopy of haematoxylin and eosin (H&E) stained samples. Additionally, when used as input data for a hybrid convolutional neural network (CNN) and random forest (RF) analyses, these yielded sensitivities, specificities and precision of 98.6 ± 0.3%, 92 ± 4% and 94 ± 5%, respectively, and an area under receiver operator characteristic (AUC) of 94 ± 2%. Our findings show the potential of O-PTIR technology as a tool to study cancer on tissue samples.

High-definition FT-IR reveals a synergistic effect on lipid accumulation in prostate cancer cells induced by a combination of X-rays and radiosensitizing drugs

Radiotherapy is one of the most commonly used cancer therapies with many benefits including low toxicity to healthy tissues. However, a major problem in radiotherapy is cancer radioresistance. To enhance the effect of this kind of therapy several approaches have been proposed such as the use of radiosensitizers. A combined treatment of radiotherapy and radiosensitizing drugs leads to a greater effect on cancer cells than anticipated from the addition of both responses (synergism). In this study, high-definition FT-IR imaging was applied to follow lipid accumulation in prostate cancer cells as a response to X-ray irradiation, radiosensitizing drugs, and a combined treatment of X-rays and the drugs. Lipid accumulation induced in the cells by an increasing X-ray dose and the presence of the drugs was analyzed using Principal Component Analysis and lipid staining. Finally, the synergistic effect of the combined therapy (X-rays and radiosensitizers) was confirmed by calculations of the integral intensity of the 2850 cm-1 band.

Deep learning analysis of mid-infrared microscopic imaging data for the diagnosis and classification of human lymphomas

The present study presents an alternative analytical workflow that combines mid-infrared (MIR) microscopic imaging and deep learning to diagnose human lymphoma and differentiate between small and large cell lymphoma. We could show that using a deep learning approach to analyze MIR hyperspectral data obtained from benign and malignant lymph node pathology results in high accuracy for correct classification, learning the distinct region of 3900 to 850 cm-1 . The accuracy is above 95% for every pair of malignant lymphoid tissue and still above 90% for the distinction between benign and malignant lymphoid tissue for binary classification. These results demonstrate that a preliminary diagnosis and subtyping of human lymphoma could be streamlined by applying a deep learning approach to analyze MIR spectroscopic data.

Tissue discrimination in head and neck cancer using image fusion of IR and optical microscopy

A regression-based fusion algorithm has been used to merge hyperspectral Fourier transform infrared (FTIR) data with an H&E image of oral squamous cell carcinoma metastases in cervical lymphoid nodal tissue. This provides insight into the success of the ratio of FTIR absorbances at 1252 cm-1 and 1285 cm-1 in discriminating between these tissue types. The success is due to absorbances at these two wavenumbers being dominated by contributions from DNA and collagen, respectively. A pixel-by-pixel fit of the fused spectra to the FTIR spectra of collagen, DNA and cytokeratin reveals the contributions of these molecules to the tissue at high spatial resolution.

Digital Histopathology by Infrared Spectroscopic Imaging

Infrared (IR) spectroscopic imaging records spatially resolved molecular vibrational spectra, enabling a comprehensive measurement of the chemical makeup and heterogeneity of biological tissues. Combining this novel contrast mechanism in microscopy with the use of artificial intelligence can transform the practice of histopathology, which currently relies largely on human examination of morphologic patterns within stained tissue. First, this review summarizes IR imaging instrumentation especially suited to histopathology, analyses of its performance, and major trends. Second, an overview of data processing methods and application of machine learning is given, with an emphasis on the emerging use of deep learning. Third, a discussion on workflows in pathology is provided, with four categories proposed based on the complexity of methods and the analytical performance needed. Last, a set of guidelines, termed experimental and analytical specifications for spectroscopic imaging in histopathology, are proposed to help standardize the diversity of approaches in this emerging area.

Adaptogenic activity of withaferin A on human cervical carcinoma cells using high-definition vibrational spectroscopic imaging

Despite invaluable advances in cervical cancer therapy, treatment regimens for recurrent or persistent cancers and low-toxicity alternative treatment options are scarce. In recent years, substances classified as adaptogens have been identified as promising drug sources for preventing and treating cancer-based diseases on their ability to attack multiple molecular targets. This paper establishes the effectiveness of inhibition of the neoplastic process by a withaferin A (WFA), an adaptogenic substance, based on an in vitro model of cervical cancer. This study explores for the first time the potential of high-definition vibrational spectroscopy methods, i.e. Fourier-transform infrared (FT-IR) and Raman spectroscopic (RS) imaging at the single-cell level to evaluate the efficacy of the adaptogenic drug. HeLa cervical cancer cells were incubated with various concentrations of WFA at different incubation times. The multimodal spectroscopic approach combined with partial least squares (PLS) regression allowed the identification of molecular changes (e.g., lipids, protein secondary structures, or nucleic acids) induced by WFA at the cellular level. The results clearly illustrate the enormous potential of WFA in inhibiting the proliferation of cervical cancer cells. WFA inhibited the growth of the studied cancer cell line in a dose-dependent manner. Such studies provide comprehensive information on the sensitivity of cells to adaptogenic drugs. This is a fundamental step towards determining the rate and nature of adaptogen-induced changes in cancer cells.

Statistical Considerations and Tools to Improve Histopathologic Protocols with Spectroscopic Imaging

Advances in infrared (IR) spectroscopic imaging instrumentation and data science now present unique opportunities for large validation studies of the concept of histopathology using spectral data. In this study, we examine the discrimination potential of IR metrics for different histologic classes to estimate the sample size needed for designing validation studies to achieve a given statistical power and statistical significance. Next, we present an automated annotation transfer tool that can allow large-scale training/validation, overcoming the limitations of sparse ground truth data with current manual approaches by providing a tool to transfer pathologist annotations from stained images to IR images across diagnostic categories. Finally, the results of a combination of supervised and unsupervised analysis provide a scheme to identify diagnostic groups/patterns and isolating pure chemical pixels for each class to better train complex histopathological models. Together, these methods provide essential tools to take advantage of the emerging capabilities to record and utilize large spectroscopic imaging datasets.

Artificial intelligence to identify genetic alterations in conventional histopathology

Precision oncology relies on the identification of targetable molecular alterations in tumor tissues. In many tumor types, a limited set of molecular tests is currently part of standard diagnostic workflows. However, universal testing for all targetable alterations, especially rare ones, is limited by the cost and availability of molecular assays. From 2017 to 2021, multiple studies have shown that artificial intelligence (AI) methods can predict the probability of specific genetic alterations directly from conventional hematoxylin and eosin (H&E) tissue slides. Although these methods are currently less accurate than gold-standard testing (e.g. immunohistochemistry, polymerase chain reaction or next-generation sequencing), they could be used as pre-screening tools to reduce the workload of genetic analyses. In this systematic literature review, we summarize the state of the art in predicting molecular alterations from H&E using AI. We found that AI methods perform reasonably well across multiple tumor types, although few algorithms have been broadly validated. In addition, we found that genetic alterations in FGFR, IDH, PIK3CA, BRAF, TP53 and DNA repair pathways are predictable from H&E in multiple tumor types, while many other genetic alterations have rarely been investigated or were only poorly predictable. Finally, we discuss the next steps for the implementation of AI-based surrogate tests in diagnostic workflows. This article is protected by copyright. All rights reserved.

Intra-operative assessment of sentinel lymph nodes for breast cancer surgery: An update

Lymph node (LN) involvement is the strongest prognostic factor in operable breast cancer (BC). Therefore, accurate assessment of LN status is essential for management of BC patients. The introduction of sentinel LN approach reduced the need for extensive axillary surgery to achieve accurate staging. However, positive sentinel LN as determined on postoperative histological examination often leads to a second axillary operation to ensure an accurate staging and that positive non-sentinel LNs are removed. Although preoperative assessment of LN has improved significantly, its accuracy remains insufficient to avoid further axillary surgery and is not sufficient to predict the status of the LN. Therefore, intraoperative evaluation of the sentinel LN to determine the need for completing lymph node dissection in case of metastasis can provide an important approach to guide BC management decision making. This article reviews the techniques available and under development for intraoperative detection of sentinel LN metastasis in BC surgery. The key features of each technique are described in detail, emphasising the benefits offered by label-free optical techniques: minimal sample preparation, high spatial resolution, and immediate on-site implementation. Optical techniques have the potential to provide a cost-effective and accurate intraoperative platform for the assessment of SLN within the operating theatre.

Exploring AdaBoost and Random Forests machine learning approaches for infrared pathology on unbalanced data sets

The use of infrared spectroscopy to augment decision-making in histopathology is a promising direction for the diagnosis of many disease types. Hyperspectral images of healthy and diseased tissue, generated by infrared spectroscopy, are used to build chemometric models that can provide objective metrics of disease state. It is important to build robust and stable models to provide confidence to the end user. The data used to develop such models can have a variety of characteristics which can pose problems to many model-building approaches. Here we have compared the performance of two machine learning algorithms - AdaBoost and Random Forests - on a variety of non-uniform data sets. Using samples of breast cancer tissue, we devised a range of training data capable of describing the problem space. Models were constructed from these training sets and their characteristics compared. In terms of separating infrared spectra of cancerous epithelium tissue from normal-associated tissue on the tissue microarray, both AdaBoost and Random Forests algorithms were shown to give excellent classification performance (over 95% accuracy) in this study. AdaBoost models were more robust when datasets with large imbalance were provided. The outcomes of this work are a measure of classification accuracy as a function of training data available, and a clear recommendation for choice of machine learning approach.

Image fusion of IR and optical microscopy for mapping of biomolecules in tissue

It is shown that a pixel-level image fusion technique can produce images that combine the spatial resolution of optical microscopy images of haematoxylin and eosin (H&E) stained tissue with the chemical information in Fourier transform infrared (FTIR) images. The fused images show minimal distortion and the higher spatial resolution of the H&E images overcomes the diffraction limit on the spatial resolution of the FTIR images. A consideration of the FTIR spectra of nucleic acids and collagen can explain the changes in contrast between non-cancerous oral epithelium and underlying stroma within fused images formed by combining an H&E stain of oral tissue with FTIR images of the tissue obtained at a number of wavenumbers.

Multi-modal image sharpening in fourier transform infrared (FTIR) microscopy

Mid-infrared Spectroscopic Imaging (MIRSI) provides spatially-resolved molecular specificity by measuring wavelength-dependent mid-infrared absorbance. Infrared microscopes use large numerical aperture objectives to obtain high-resolution images of heterogeneous samples. However, the optical resolution is fundamentally diffraction-limited, and therefore wavelength-dependent. This significantly limits resolution in infrared microscopy, which relies on long wavelengths (2.5 μm to 12.5 μm) for molecular specificity. The resolution is particularly restrictive in biomedical and materials applications, where molecular information is encoded in the fingerprint region (6 μm to 12 μm), limiting the maximum resolving power to between 3 μm and 6 μm. We present an unsupervised curvelet-based image fusion method that overcomes limitations in spatial resolution by augmenting infrared images with label-free visible microscopy. We demonstrate the effectiveness of this approach by fusing images of breast and ovarian tumor biopsies acquired using both infrared and dark-field microscopy. The proposed fusion algorithm generates a hyperspectral dataset that has both high spatial resolution and good molecular contrast. We validate this technique using multiple standard approaches and through comparisons to super-resolved experimentally measured photothermal spectroscopic images. We also propose a novel comparison method based on tissue classification accuracy.

Simultaneous Characterization of Implant Wear and Tribocorrosion Debris within Its Corresponding Tissue Response Using Infrared Chemical Imaging

Biotribology is one of the key branches in the field of artificial joint development. Wear and corrosion are among fundamental processes which cause material loss in a joint biotribological system; the characteristics of wear and corrosion debris are central to determining the in vivo bioreactivity. Much effort has been made elucidating the debris-induced tissue responses. However, due to the complexity of the biological environment of the artificial joint, as well as a lack of effective imaging tools, there is still very little understanding of the size, composition, and concentration of the particles needed to trigger adverse local tissue reactions, including periprosthetic osteolysis. Fourier transform infrared spectroscopic imaging (FTIR-I) provides fast biochemical composition analysis in the direct context of underlying physiological conditions with micron-level spatial resolution, and minimal additional sample preparation in conjunction with the standard histopathological analysis workflow. In this study, we have demonstrated that FTIR-I can be utilized to accurately identify fine polyethylene debris accumulation in macrophages that is not achievable using conventional or polarized light microscope with histological staining. Further, a major tribocorrosion product, chromium phosphate, can be characterized within its histological milieu, while simultaneously identifying the involved immune cell such as macrophages and lymphocytes. In addition, we have shown the different spectral features of particle-laden macrophages through image clustering analysis. The presence of particle composition variance inside macrophages could shed light on debris evolution after detachment from the implant surface. The success of applying FTIR-I in the characterization of prosthetic debris within their biological context may very well open a new avenue of research in the orthopedics community.

Influence of interference effects on the spectral quality and histological classification by FT-IR imaging in transflection geometry

Infrared (IR) imaging can be used for fast, accurate and non-destructive pathology recognition of biopsies when supported by machine learning algorithms. Transflection mode of measurements has the potential to be translated into the clinic due to economic reasons of large-scale imaging with the need for inexpensive substrates. Unfortunately, in this mode spectral distortions originating from light interference appear. Due to this fact transmission measurement mode is more frequently used in pathology recognition. Nevertheless, this measurement mode also is not devoid of spectral distortion effects like scattering. However, this effect is better understood and there are preprocessing algorithms to minimize it. In this work, we investigated the influence of interference effects on spectral quality of pancreatic tissues measured in transmission and transflection mode with Fourier tranform IR (FT-IR) microscopy using samples embedded with and without paraffin. The removal of paraffin leads to an altered magnitude of interference in transflection and provides a platform for a detailed analysis of its effect on the spectra of biological material, since the same sample is measured with different interference conditions. Moreover, the potential of transflection mode measurements in histological classification of analyzed samples was investigated and compared with classification results for transmission mode.

Translation of an esophagus histopathological FT-IR imaging model to a fast quantum cascade laser modality

The technical progress in fast quantum cascade laser (QCL) microscopy offers a platform where chemical imaging becomes feasible for clinical diagnostics. QCL systems allow the integration of previously developed FT-IR-based pathology recognition models in a faster workflow. The translation of such models requires a systematic approach, focusing only on the spectral frequencies that carry crucial information for discrimination of pathologic features. In this study, we optimize an FT-IR-based histopathological method for esophageal cancer detection to work with a QCL system. We explore whether the classifier's performance is affected by paraffin presence from tissue blocks compared to removing it chemically. Working with paraffin-embedded samples reduces preprocessing time in the lab and allows samples to be archived after analysis. Moreover, we test, whether the creation of a QCL model requires a preestablished FTIR model or can be optimized using solely QCL measurements.

Comparison between high definition FT-IR, Raman and AFM-IR for subcellular chemical imaging of cholesteryl esters in prostate cancer cells

The family of vibrational spectroscopic imaging techniques grows every few years and there is a need to compare and contrast new modalities with the better understood ones, especially in the case of demanding biological samples. Three vibrational spectroscopy techniques (high definition Fourier-transform infrared [FT-IR], Raman and atomic force microscopy infrared [AFM-IR]) were applied for subcellular chemical imaging of cholesteryl esters in PC-3 prostate cancer cells. The techniques were compared and contrasted in terms of image quality, spectral pattern and chemical information. All tested techniques were found to be useful in chemical imaging of cholesterol derivatives in cancer cells. The results obtained from FT-IR and Raman imaging showed to be comparable, whereas those achieved from AFM-IR study exhibited higher spectral heterogeneity. It confirms AFM-IR method as a powerful tool in local chemical imaging of cells at the nanoscale level. Furthermore, due to polarization effect, p-polarized AFM-IR spectra showed strong enhancement of lipid bands when compared to FT-IR.

New Technologies to Image Tumors

The premise of this book is the importance of the tumor microenvironment (TME). Until recently, most research on and clinical attention to cancer biology, diagnosis, and prognosis were focused on the malignant (or premalignant) cellular compartment that could be readily appreciated using standard morphology-based imaging.

Noise-free simulation of an FT-IR imaging hyperspectral dataset of pancreatic biopsy core bound by experiment

A noise-free hyperspectral FT-IR imaging dataset of a pancreatic tissue core was simulated based on experimental data that allows to test the performance of various data analysis and processing algorithms. A set of experimental noise levels was also added and used for denoising approaches comparison, which due to the noise-free reference signal enables to truly observe signal distortion caused by different approaches.

Digital Staining of High-Definition Fourier Transform Infrared (FT-IR) Images Using Deep Learning

Histological stains, such as hematoxylin and eosin (H&E), are routinely used in clinical diagnosis and research. While these labels offer a high degree of specificity, throughput is limited by the need for multiple samples. Traditional histology stains, such as immunohistochemical labels, also rely only on protein expression and cannot quantify small molecules and metabolites that may aid in diagnosis. Finally, chemical stains and dyes permanently alter the tissue, making downstream analysis impossible. Fourier transform infrared (FT-IR) spectroscopic imaging has shown promise for label-free characterization of important tissue phenotypes and can bypass the need for many chemical labels. Fourier transform infrared classification commonly leverages supervised learning, requiring human annotation that is tedious and prone to errors. One alternative is digital staining, which leverages machine learning to map IR spectra to a corresponding chemical stain. This replaces human annotation with computer-aided alignment. Previous work relies on alignment of adjacent serial tissue sections. Since the tissue samples are not identical at the cellular level, this technique cannot be applied to high-definition FT-IR images. In this paper, we demonstrate that cellular-level mapping can be accomplished using identical samples for both FT-IR and chemical labels. In addition, higher-resolution results can be achieved using a deep convolutional neural network that integrates spatial and spectral features.

A fully automated, faster noise rejection approach to increasing the analytical capability of chemical imaging for digital histopathology

Chemical hyperspectral imaging (HSI) data is naturally high dimensional and large. There are thus inherent manual trade-offs in acquisition time, and the quality of data. Minimum Noise Fraction (MNF) developed by Green et al. [1] has been extensively studied as a method for noise removal in HSI data. It too, however entails a manual speed-accuracy trade-off, namely the process of manually selecting the relevant bands in the MNF space. This process currently takes roughly around a month’s time for acquiring and pre-processing an entire TMA with acceptable signal to noise ratio. We present three approaches termed ‘Fast MNF’, ‘Approx MNF’ and ‘Rand MNF’ where the computational time of the algorithm is reduced, as well as the entire process of band selection is fully automated. This automated approach is shown to perform at the same level of accuracy as MNF with now large speedup factors, resulting in the same task to be accomplished in hours. The different approximations produced by the three algorithms, show the reconstruction accuracy vs storage (50×) and runtime speed (60×) trade-off. We apply the approach for automating the denoising of different tissue histology samples, in which the accuracy of classification (differentiating between the different histologic and pathologic classes) strongly depends on the SNR (signal to noise ratio) of recovered data. Therefore, we also compare the effect of the proposed denoising algorithms on classification accuracy. Since denoising HSI data is done unsupervised, we also use a metric that assesses the quality of denoising in the image domain between the noisy and denoised image in the absence of ground truth.

High and ultra-high definition of infrared spectral histopathology gives an insight into chemical environment of lung metastases in breast cancer

Using high definition (HD) and ultra-high definition (UHD) of Fourier-transform infrared (FTIR) spectroscopic imaging, we characterized spectrally pulmonary metastases in a murine model of breast cancer comparing them with histopathological results (Hematoxylin and eosin [H&E] staining). This comparison showed excellent agreement between the methods in case of localization of metastases with size below 1 mm and revealed that label-free HD and UHD IR spectral histopathology distinguish the type of neoplastic cells. We primary focused on differentiation between metastatic foci in the pleural cavity from cancer cells present in lung parenchyma and inflamed cells present in extracellular matrix of lungs due to growing of advanced metastases. In addition, a combination of unsupervised clustering and IR imaging indicated the high sensitivity of FTIR spectroscopy to identify chemical features of small macrometastases located under the pleural cavity and during epithelial-mesenchymal transition. FTIR-based spectral histopathology was proved to detect not only phases of breast cancer metastasis to lungs but also to differentiate various origins of metastases seeded from breast cancer.

Phase-sensitive lock-in detection for high-contrast mid-infrared photothermal imaging with sub-diffraction limited resolution

Imaging of the phase output of a lock-in amplifier in mid-infrared photothermal vibrational microscopy is demonstrated for the first time in combination with nonlinear demodulation. In general, thermal blurring and heat transport phenomena contribute to the resolution and sensitivity of mid-infrared photothermal imaging. For heterogeneous samples with multiple absorbing features, if imaged in a spectral regime of comparable absorption with their embedding medium, it is demonstrated that differentiation with high contrast is achieved in complementary imaging of the phase signal obtained from a lock-in amplifier compared to standard imaging of the photothermal amplitude signal. Specifically, by investigating the relative contribution of the out-of-phase lock-in signal, information based on changes in the rate of heat transport can be extracted, and inhomogeneities in the thermal diffusion properties across the sample plane can be mapped with high sensitivity and sub-diffraction limited resolution. Under these imaging conditions, wavenumber regimes can be identified in which the thermal diffusion contributions are minimized and an enhancement of the spatial resolution beyond the diffraction limited spot size of the probe beam in the corresponding phase images is achieved. By combining relative diffusive phase imaging with nonlinear demodulation at the second harmonic, it is demonstrated that 1-μm-size melamine beads embedded in a thin layer of 4-octyl-4'-cyanobiphenyl (8CB) liquid crystal can be detected with a 1.3-μm spatial full-width at half-maximum (FWHM) resolution. Thus, imaging with a resolving power that exceeds the probe diffraction limited spot size by a factor of 2.5 is presented, which paves the route towards super-resolution, label-free imaging in the mid-infrared.

Denoising influence on discrete frequency classification results for quantum cascade laser based infrared microscopy

Currently, there is great interest in bringing the application of IR spectroscopy into the clinic. This however will require a significant reduction in measurement time as Fourier Transform Infrared (FT-IR) imaging takes hours to days to scan a clinically relevant specimen. A potential remedy for this issue is the use of Quantum Cascade Laser Infrared (QCL IR) microscopy performed in Discrete Frequency (DF) mode for maximum speed gain. This gain could be furthermore improved by applying a proper denoising algorithm that takes into account the specific data structure. We have recently compared spectral and spatial denoising techniques in the context of Fourier Transform IR (FT-IR) imaging and showed that the optimal methods depend heavily on the exact data structure. In general multivariate denoising methods such as Principal Component Analysis (PCA) and Minimum Noise Fraction (MNF) are the most effective for a dataset containing multiple bands. Histologic classification of QCL IR images of pancreatic tissue using Random Forest was therefore performed to investigate which denoising schemes are the most optimal for such experimental data structure. This work is the first to show the effects of denoising on classification accuracy of QCL data and is likely to be transferable to other QCL microscopes and other modalities using DF imaging, e.g. AFM-IR or CARS/SRS imaging.

SIproc: an open-source biomedical data processing platform for large hyperspectral images

There has recently been significant interest within the vibrational spectroscopy community to apply quantitative spectroscopic imaging techniques to histology and clinical diagnosis. However, many of the proposed methods require collecting spectroscopic images that have a similar region size and resolution to the corresponding histological images. Since spectroscopic images contain significantly more spectral samples than traditional histology, the resulting data sets can approach hundreds of gigabytes to terabytes in size. This makes them difficult to store and process, and the tools available to researchers for handling large spectroscopic data sets are limited. Fundamental mathematical tools, such as MATLAB, Octave, and SciPy, are extremely powerful but require that the data be stored in fast memory. This memory limitation becomes impractical for even modestly sized histological images, which can be hundreds of gigabytes in size. In this paper, we propose an open-source toolkit designed to perform out-of-core processing of hyperspectral images. By taking advantage of graphical processing unit (GPU) computing combined with adaptive data streaming, our software alleviates common workstation memory limitations while achieving better performance than existing applications.

Amalgam tattoo versus melanocytic neoplasm - Differential diagnosis of dark pigmented oral mucosa lesions using infrared spectroscopy

Background

Dark pigmented lesions of the oral mucosa can represent a major diagnostic challenge. A biopsy is usually required to determine the nature of such intraoral discolorations. This study investigates the potential use of infrared spectroscopy for differential diagnosis of amalgam tattoos versus benign or malignant melanocytic neoplasms.

Materials and methods

For this retrospective study, formalin-fixed paraffin-embedded tissue (FFPE) specimens of dark pigmented lesions concerning the oral mucosa or the lip were investigated using mid infrared spectroscopy. The samples were chosen from patients who had undergone a mucosal biopsy at the University Hospital Innsbruck (Austria) between the years 2000 and 2017. Principal component analysis was used for data exploration. Evaluation was based on the superimposition of the recorded spectra and the corresponding histologic slides.

Results

In total, 22 FFPE specimens were analyzed. Clear differences were found between amalgam and non-amalgam samples. A general weakening of the penetrating infrared radiation allowed for unspecific discrimination between these two classes. An overall accuracy in predicting the correct class of 95.24% was achieved.

Conclusion

Infrared spectroscopy appears to be a suitable technique to differentiate between amalgam tattoos and melanocytic lesions in FFPE samples. It could potentially be applied in vivo, too, serving as a non-invasive diagnostic tool for intraoral dark pigmented lesions.

Comparison of spectral and spatial denoising techniques in the context of High Definition FT-IR imaging hyperspectral data

The recent emergence of High Definition (HD) FT-IR and Quantum Cascade Laser (QCL) Microscopes elevated the IR imaging field very close to clinical timescales. However, the speed of acquisition and data quality are still the critical factors in reaching the clinic. Denoising offers aide in both aspects if performed properly. However, there is a lack of a direct comparison of the efficiency of denoising techniques in IR imaging in general. To achieve such comparison within a rigorous framework and obtaining the critical information about signal loss, a simulated dataset strongly bound by experimental parameters was created. Using experimental structural and spectral information and experimental noise levels data as an input for the simulation, a direct comparison of spatial (Fourier transform, Mean Filter, Weighted Mean Filter, Gauss Filter, Median Filter, spatial Wavelets and Deep Neural Networks) and spectral (Savitzky-Golay, Fourier transform, Principal Component Analysis, Minimum Noise Fraction and spectral Wavelets) denoising schemes was enabled. All of these techniques were compared on the simulated dataset, taking into account SNR gain, signal distortion and sensitivity to tuning parameters as comparison metrics. Later, the best techniques were applied to experimental data for validation. The results presented here clearly show the benefit of using hyperspectral denoising schemes such as PCA and MNF which outperform other methods.

Multimodal Chemical Analysis of the Brain by High Mass Resolution Mass Spectrometry and Infrared Spectroscopic Imaging

The brain functions through chemical interactions between many different cell types, including neurons and glia. Acquiring comprehensive information on complex, heterogeneous systems requires multiple analytical tools, each of which have unique chemical specificity and spatial resolution. Multimodal imaging generates complementary chemical information via spatially localized molecular maps, ideally from the same sample, but requires method enhancements that span from data acquisition to interpretation. We devised a protocol for performing matrix-assisted laser desorption/ionization (MALDI)-Fourier transform ion cyclotron resonance-mass spectrometry imaging (MSI), followed by infrared (IR) spectroscopic imaging on the same specimen. Multimodal measurements from the same tissue provide precise spatial alignment between modalities, enabling more advanced image processing such as image fusion and sharpening. Performing MSI first produces higher quality data from each technique compared to performing IR imaging before MSI. The difference is likely due to fixing the tissue section during MALDI matrix removal, thereby preventing analyte degradation occurring during IR imaging from an unfixed specimen. Leveraging the unique capabilities of each modality, we utilized pan sharpening of MS (mass spectrometry) ion images with selected bands from IR spectroscopy and midlevel data fusion. In comparison to sharpening with histological images, pan sharpening can employ a plethora of IR bands, producing sharpened MS images while retaining the fidelity of the initial ion images. Using Laplacian pyramid sharpening, we determine the localization of several lipids present within the hippocampus with high mass accuracy at 5 μm pixel widths. Further, through midlevel data fusion of the imaging data sets combined with k-means clustering, the combined data set discriminates between additional anatomical structures unrecognized by the individual imaging approaches. Significant differences between molecular ion abundances are detected between relevant structures within the hippocampus, such as the CA1 and CA3 regions. Our methodology provides high quality multiplex and multimodal chemical imaging of the same tissue sample, enabling more advanced data processing and analysis routines.

Mid-infrared spectral microimaging of inflammatory skin lesions

Skin is one of the most important organs of the human body because of its characteristics and functions. There are many alterations, either pathological or physiological, that can disturb its functioning. However, at present all methods used to investigate skin diseases, non-invasive or invasive, are based on clinical examinations by physicians. Thus, diagnosis, prognosis and therapeutic management rely on the expertise of the practitioner, the quality of the method and the accessibility of distinctive morphological characteristics of each lesion. To overcome the high sensitivity of these parameters, techniques based on more objective criteria must be explored. Vibrational spectroscopy has become as a key technique for tissue analysis in the biomedical research field. Based on a non-destructive light/matter interaction, this tool provides information about specific molecular structure and composition of the analyzed sample, thus relating to its precise physiopathological state and permitting to distinguish lesional from normal tissues. This label-free optical method can be performed directly on the paraffin-embedded tissue sections without chemical dewaxing. In this study, the potential of the infrared microspectroscopy, combined with data classification methods was demonstrated, to characterize at the tissular level different types of inflammatory skin lesions, and this independently from conventional histopathology.

Simultaneous cancer and tumor microenvironment subtyping using confocal infrared microscopy for all-digital molecular histopathology

Cancer alters both the morphological and the biochemical properties of multiple cell types in a tissue. Generally, the morphology of epithelial cells is practical for routine disease diagnoses. Here, infrared spectroscopic imaging biochemically characterizes breast cancer, both epithelial cells and the tumor-associated microenvironment. Unfortunately, conventional spectral analyses are slow. Hence, we designed and built a laser confocal microscope that demonstrates a high signal-to-noise ratio for confident diagnoses. The instrument cuts down imaging time from days to minutes, making the technology feasible for research and clinical translation. Finally, automated human breast cancer biopsy imaging is reported in ∼1 hour, paving the way for routine research into the total tumor (epithelial plus microenvironment) properties and rapid, label-free diagnoses.

Histopathology based on spatial patterns of epithelial cells is the gold standard for clinical diagnoses and research in carcinomas; although known to be important, the tissue microenvironment is not readily used due to complex and subjective interpretation with existing tools. Here, we demonstrate accurate subtyping from molecular properties of epithelial cells using emerging high-definition Fourier transform infrared (HD FT-IR) spectroscopic imaging combined with machine learning algorithms. In addition to detecting four epithelial subtypes, we simultaneously delineate three stromal subtypes that characterize breast tumors. While FT-IR imaging data enable fully digital pathology with rich information content, the long spectral scanning times required for signal averaging and processing make the technology impractical for routine research or clinical use. Hence, we developed a confocal design in which refractive IR optics are designed to provide high-definition, rapid spatial scanning and discrete spectral tuning using a quantum cascade laser (QCL) source. This instrument provides simultaneously high resolving power (2-μm pixel size) and high signal-to-noise ratio (SNR) (>1,300), providing a speed increase of ∼50-fold for obtaining classified results compared with present imaging spectrometers. We demonstrate spectral fidelity and interinstrument operability of our developed instrument by accurate analysis of a 100-case breast tissue set that was analyzed in a day, considerably speeding research. Clinical breast biopsies typical of a patients’ caseload are analyzed in ∼1 hour. This study paves the way for comprehensive tumor-microenvironment analyses in feasible time periods, presenting a critical step in practical label-free molecular histopathology.

FT-IR- and Raman-based biochemical profiling of the early stage of pulmonary metastasis of breast cancer in mice

The combination of FT-IR and Raman spectroscopies allowed the biochemical profiling of lungs and definition of the spectroscopic biomarkers of the early stage of pulmonary metastasis of breast cancer.

Emerging Themes in Image Informatics and Molecular Analysis for Digital Pathology

Pathology is essential for research in disease and development, as well as for clinical decision making. For more than 100 years, pathology practice has involved analyzing images of stained, thin tissue sections by a trained human using an optical microscope. Technological advances are now driving major changes in this paradigm toward digital pathology (DP). The digital transformation of pathology goes beyond recording, archiving, and retrieving images, providing new computational tools to inform better decision making for precision medicine. First, we discuss some emerging innovations in both computational image analytics and imaging instrumentation in DP. Second, we discuss molecular contrast in pathology. Molecular DP has traditionally been an extension of pathology with molecularly specific dyes. Label-free, spectroscopic images are rapidly emerging as another important information source, and we describe the benefits and potential of this evolution. Third, we describe multimodal DP, which is enabled by computational algorithms and combines the best characteristics of structural and molecular pathology. Finally, we provide examples of application areas in telepathology, education, and precision medicine. We conclude by discussing challenges and emerging opportunities in this area.

Data fusion of near-infrared and mid-infrared spectra for identification of rhubarb

Rhubarb has different medicinal efficacy to official rhubarb and may affect the clinical medication safety. In order to guarantee the quality of rhubarb, we established a method to distinguish unofficial rhubarbs. 52 official and unofficial rhubarb samples were analyzed using near-infrared (NIR) spectroscopy and mid-infrared (MIR) spectroscopy for classification. The feature vectors, which were selected by wavelet compression (WC) and interval partial least squares (iPLS) from NIR, MIR spectra, were fused together for identifying rhubarb samples. Partial least squares-discriminant analysis (PLS-DA), soft independent modeling of class analogies (SIMCA), support vector machine (SVM) and artificial neural network (ANN) were compared for classifying rhubarb. The use of data fusion strategies improved the classification model and allowed correct classification of all the samples.

Can mid-infrared biomedical spectroscopy of cells, fluids and tissue aid improvements in cancer survival? A patient paradigm

This review will take a fresh approach from the patient perspective; offering insight into the applications of mid-infrared biomedical spectroscopy in a scenario whereby the patient presents with non-specific symptoms and via an extensive diagnostic process multiple lesions are discovered but no clear sign of the primary tumour; a condition known as cancer of unknown primary (CUP). With very limited options to diagnose the cancer origin, treatment options are likely to be ineffective and prognosis is consequentially very poor. CUP has not yet been targeted by infrared biospectroscopy, however, this timely, concise dissemination will focus on a series of research highlights and breakthroughs from the field for the management of a variety of cancer-related diseases - many examples of which have occurred within this year alone. The case for integration of mid-infrared (MIR) technology into clinical practice will be demonstrated largely via diagnostic, but also therapeutic and prognostic avenues by means of including cytological, bio-fluid and tissue analysis. The review is structured around CUP but is relevant for all cancer diagnoses. Infrared spectroscopy is fast developing a reputation as a valid and powerful tool for the detection and diagnosis of cancer using a variety of sample formats. The technology will produce data and tools that are designed to complement routine clinical practice; enhancing the ability of the clinician to make a reliable and non-subjective decision and enabling decreased levels of mortality and morbidity and gains in patient quality of life.

High-resolution FTIR imaging of colon tissues for elucidation of individual cellular and histopathological features

Comparison of spectral-histopathological features of a colon tissue measured using a conventional (5.5 μm × 5.5 μm, left) and a high-magnification (1.1 μm × 1.1 μm, right) FTIR imaging system with respect to HE stained tissue (middle).