Roadmap on optical sensors

Optical sensors and sensing technologies are playing a more and more important role in our modern world. From micro-probes to large devices used in such diverse areas like medical diagnosis, defence, monitoring of industrial and environmental conditions, optics can be used in a variety of ways to achieve compact, low cost, stand-off sensing with extreme sensitivity and selectivity. Actually, the challenges to the design and functioning of an optical sensor for a particular application requires intimate knowledge of the optical, material, and environmental properties that can affect its performance. This roadmap on optical sensors addresses different technologies and application areas. It is constituted by twelve contributions authored by world-leading experts, providing insight into the current state-of-the-art and the challenges their respective fields face. Two articles address the area of optical fibre sensors, encompassing both conventional and specialty optical fibres. Several other articles are dedicated to laser-based sensors, micro- and nano-engineered sensors, whispering-gallery mode and plasmonic sensors. The use of optical sensors in chemical, biological and biomedical areas is discussed in some other papers. Different approaches required to satisfy applications at visible, infrared and THz spectral regions are also discussed.

Distinguishing Asphyxia from Sudden Cardiac Death as the Cause of Death from the Lung Tissues of Rats and Humans Using Fourier Transform Infrared Spectroscopy

The ability to determine asphyxia as a cause of death is important in forensic practice and helps us to judge whether a case is criminal. However, in some cases where the deceased has underlying heart disease, death by asphyxia cannot be determined by traditional autopsy and morphological observation under a microscope because there are no specific morphological features for either asphyxia or sudden cardiac death (SCD). Here, Fourier transform infrared (FTIR) spectroscopy was employed to distinguish asphyxia from SCD. A total of 40 lung tissues (collected at 0 h and 24 h postmortem) from 20 rats (10 died from asphyxia and 10 died from SCD) and 16 human lung tissues from 16 real cases were used for spectral data acquisition. After data preprocessing, 2675 spectra from rat lung tissues and 1526 spectra from human lung tissues were obtained for subsequent analysis. First, we found that there were biochemical differences in the rat lung tissues between the two causes of death by principal component analysis and partial least-squares discriminant analysis (PLS-DA), which were related to alterations in lipids, proteins, and nucleic acids. In addition, a PLS-DA classification model can be built to distinguish asphyxia from SCD. Second, based on the spectral data obtained from lung tissues allowed to decompose for 24 h, we could still distinguish asphyxia from SCD even when decomposition occurred in animal models. Nine important spectral features that contributed to the discrimination in the animal experiment were selected and further analyzed. Third, 7 of the 9 differential spectral features were also found to be significantly different in human lung tissues from 16 real cases. A support vector machine model was finally built by using the seven variables to distinguish asphyxia from SCD in the human samples. Compared with the linear PLS-DA model, its accuracy was significantly improved to 0.798, and the correct rate of determining the cause of death was 100%. This study shows the application potential of FTIR spectroscopy for exploring the subtle biochemical differences resulting from different death processes and determining the cause of death even after decomposition.

Exploring AdaBoost and Random Forests machine learning approaches for infrared pathology on unbalanced data sets

The use of infrared spectroscopy to augment decision-making in histopathology is a promising direction for the diagnosis of many disease types. Hyperspectral images of healthy and diseased tissue, generated by infrared spectroscopy, are used to build chemometric models that can provide objective metrics of disease state. It is important to build robust and stable models to provide confidence to the end user. The data used to develop such models can have a variety of characteristics which can pose problems to many model-building approaches. Here we have compared the performance of two machine learning algorithms - AdaBoost and Random Forests - on a variety of non-uniform data sets. Using samples of breast cancer tissue, we devised a range of training data capable of describing the problem space. Models were constructed from these training sets and their characteristics compared. In terms of separating infrared spectra of cancerous epithelium tissue from normal-associated tissue on the tissue microarray, both AdaBoost and Random Forests algorithms were shown to give excellent classification performance (over 95% accuracy) in this study. AdaBoost models were more robust when datasets with large imbalance were provided. The outcomes of this work are a measure of classification accuracy as a function of training data available, and a clear recommendation for choice of machine learning approach.

Using hyperspectral imaging automatic classification of gastric cancer grading with a shallow residual network

The gastric cancer grading of patients determines their clinical treatment plan. We use hyperspectral imaging (HSI) gastric cancer section data to automatically classify the three different cancer grades (low grade, intermediate grade, and high grade) and healthy tissue. This paper proposed the use of HSI data combined with a shallow residual network (SR-Net) as the classifier. We collected hyperspectral data from gastric sections of 30 participants, with the wavelength range of hyperspectral data being 374 nm to 990 nm. We compared the classification results between hyperspectral data and color images. The results show that using hyperspectral data and a SR-Net an average classification accuracy of 91.44% could be achieved, which is 13.87% higher than that of the color image. In addition, we applied a modified SR-Net incorporated direct down-sampling, asymmetric filters, and global average pooling to reduce the parameters and floating-point operations. Compared with the regular residual network with the same number of blocks, the floating-point operations of a SR-Net are one order of magnitude less. The experimental results show that hyperspectral data with a SR-Net can achieve cutting-edge performance with minimum computational cost and therefore have potential in the study of gastric cancer grading.

Spectral histopathology of the lung: A review of two large studies

This paper summarizes results from two large lung cancer studies comprising over 700 samples that demonstrate the ability of spectral histopathology (SHP) to distinguish cancerous tissue regions from normal tissue, to differentiate benign lesions from normal tissue and cancerous lesions, and to classify lung cancer types. Furthermore, malignancy-associated changes can be identified in cancer-adjacent normal tissue. The ability to differentiate a multitude of normal cells and tissue types allow SHP to identify tumor margins and immune cell infiltration. Finally, SHP easily distinguishes small cell lung cancer (SCLC) from non-SCLC (NSCLC) and provides a further differentiation of NSCLC into adenocarcinomas and squamous cell carcinomas with an accuracy comparable of classical histopathology combined with immunohistochemistry. Case studies are presented that demonstrates that SHP can resolve interobserver discrepancies in standard histopathology.

Digital Staining of High-Definition Fourier Transform Infrared (FT-IR) Images Using Deep Learning

Histological stains, such as hematoxylin and eosin (H&E), are routinely used in clinical diagnosis and research. While these labels offer a high degree of specificity, throughput is limited by the need for multiple samples. Traditional histology stains, such as immunohistochemical labels, also rely only on protein expression and cannot quantify small molecules and metabolites that may aid in diagnosis. Finally, chemical stains and dyes permanently alter the tissue, making downstream analysis impossible. Fourier transform infrared (FT-IR) spectroscopic imaging has shown promise for label-free characterization of important tissue phenotypes and can bypass the need for many chemical labels. Fourier transform infrared classification commonly leverages supervised learning, requiring human annotation that is tedious and prone to errors. One alternative is digital staining, which leverages machine learning to map IR spectra to a corresponding chemical stain. This replaces human annotation with computer-aided alignment. Previous work relies on alignment of adjacent serial tissue sections. Since the tissue samples are not identical at the cellular level, this technique cannot be applied to high-definition FT-IR images. In this paper, we demonstrate that cellular-level mapping can be accomplished using identical samples for both FT-IR and chemical labels. In addition, higher-resolution results can be achieved using a deep convolutional neural network that integrates spatial and spectral features.

New insights into spectral histopathology: infrared-based scoring of tumour aggressiveness of squamous cell lung carcinomas

Spectral histopathology, based on infrared interrogation of tissue sections, proved a promising tool for helping pathologists in characterizing histological structures in a quantitative and automatic manner.

Spectral histopathology, based on infrared interrogation of tissue sections, proved a promising tool for helping pathologists in characterizing histological structures in a quantitative and automatic manner. In cancer diagnosis, the use of chemometric methods permits establishing numerical models able to detect cancer cells and to characterize their tissular environment. In this study, we focused on exploiting multivariate infrared data to score the tumour aggressiveness in preneoplastic lesions and squamous cell lung carcinomas. These lesions present a wide range of aggressive phenotypes; it is also possible to encounter cases with various degrees of aggressiveness within the same lesion. Implementing an infrared-based approach for a more precise histological determination of the tumour aggressiveness should arouse interest among pathologists with direct benefits for patient care. In this study, the methodology was developed from a set of samples including all degrees of tumour aggressiveness and by constructing a chain of data processing steps for an automated analysis of tissues currently manipulated in routine histopathology.

Deep learning for FTIR histology: leveraging spatial and spectral features with convolutional neural networks

Current methods for cancer detection rely on tissue biopsy, chemical labeling/staining, and examination of the tissue by a pathologist. Though these methods continue to remain the gold standard, they are non-quantitative and susceptible to human error. Fourier transform infrared (FTIR) spectroscopic imaging has shown potential as a quantitative alternative to traditional histology. However, identification of histological components requires reliable classification based on molecular spectra, which are susceptible to artifacts introduced by noise and scattering. Several tissue types, particularly in heterogeneous tissue regions, tend to confound traditional classification methods. Convolutional neural networks (CNNs) are the current state-of-the-art in image classification, providing the ability to learn spatial characteristics of images. In this paper, we demonstrate that CNNs with architectures designed to process both spectral and spatial information can significantly improve classifier performance over per-pixel spectral classification. We report classification results after applying CNNs to data from tissue microarrays (TMAs) to identify six major cellular and acellular constituents of tissue, namely adipocytes, blood, collagen, epithelium, necrosis, and myofibroblasts. Experimental results show that the use of spatial information in addition to the spectral information brings significant improvements in the classifier performance and allows classification of cellular subtypes, such as adipocytes, that exhibit minimal chemical information but have distinct spatial characteristics. This work demonstrates the application and efficiency of deep learning algorithms in improving the diagnostic techniques in clinical and research activities related to cancer.

Resolving Interobserver Discrepancies in Lung Cancer Diagnoses by Spectral Histopathology

This paper reports the results of a collaborative lung cancer study between City of Hope Cancer Center (Duarte, California) and CIRECA, LLC (Cambridge, Massachusetts), comprising 328 samples from 249 patients, that used an optical technique known as spectral histopathology (SHP) for tissue classification. Because SHP is based on a physical measurement, it renders diagnoses on a more objective and reproducible basis than methods based on assessing cell morphology and tissue architecture. This report demonstrates that SHP provides distinction of adenocarcinomas from squamous cell carcinomas of the lung with an accuracy comparable to that of immunohistochemistry and highly reliable classification of adenosquamous carcinoma. Furthermore, this report shows that SHP can be used to resolve interobserver differences in lung pathology. Spectral histopathology is based on the detection of changes in biochemical composition, rather than morphologic features, and is therefore more akin to methods such as matrix-assisted laser desorption ionization time-of-flight mass spectrometry imaging. Both matrix-assisted laser desorption ionization time-of-flight mass spectrometry and SHP imaging modalities demonstrate that changes in tissue morphologic features observed in classical pathology are accompanied by, and may be correlated to, changes in the biochemical composition at the cellular level. Thus, these imaging methods provide novel insight into biochemical changes due to disease.

Optimizing decision tree structures for spectral histopathology (SHP)

This paper reviews methods to arrive at optimum decision tree or label tree structures to analyze large SHP datasets. Supervised methods of analysis can utilize either sequential or (flat) multi-classifiers depending on the variance in the data, and on the number of spectral classes to be distinguished. For small number of spectral classes, multi-classifiers have been used in the past, but for the analysis of datasets containing large numbers (∼20) of disease or tissue types, mixed decision tree structures were found to be advantageous. In these mixed structures, discrimination into classes and subclasses is achieved via hierarchical decision/label tree structures.

Infrared spectral histopathology using haematoxylin and eosin (H&E) stained glass slides: a major step forward towards clinical translation

Infrared spectral histopathology has shown great promise as an important diagnostic tool, with the potential to complement current pathological methods. While promising, clinical translation has been hindered by the impracticalities of using infrared transmissive substrates which are both fragile and prohibitively very expensive. Recently, glass has been proposed as a potential replacement which, although largely opaque in the infrared, allows unrestricted access to the high wavenumber region (2500-3800 cm^-1). Recent studies using unstained tissue on glass have shown that despite utilising only the amide A band, good discrimination between histological classes could be achieved, and suggest the potential of discriminating between normal and malignant tissue. However unstained tissue on glass has the potential to disrupt the pathologist workflow, since it needs to be stained following infrared chemical imaging. In light of this, we report on the very first infrared Spectral Histopathology SHP study utilising coverslipped H&E stained tissue on glass using samples as received from the pathologist. In this paper we present a rigorous study using results obtained from an extended patient sample set consisting of 182 prostate tissue cores obtained from 100 different patients, on 18 separate H&E slides. Utilising a Random Forest classification model we demonstrate that we can rapidly classify four classes of histology of an independent test set with a high degree of accuracy (>90%). We investigate different degrees of staining using nine separate prostate serial sections, and demonstrate that we discriminate on biomarkers rather than the presence of the stain. Finally, using a four-class model we show that we can discriminate normal epithelium, malignant epithelium, normal stroma and cancer associated stroma with classification accuracies over 95%.

Early diagnosis of Alzheimer's disease using infrared spectroscopy of isolated blood samples followed by multivariate analyses

Alzheimer's disease (AD) is the most common cause of dementia, particularly in the elderly. The disease is characterized by cognitive decline that typically starts with insidious memory loss and progresses relentlessly to produce global impairment of all higher cortical functions. Due to better living conditions and health facilities in developed countries, which result in higher overall life spans, these countries report upward trends of AD among their populations. There are, however, no specific diagnostic tests for AD and clinical diagnosis is especially difficult in the earliest stages of the disease. Early diagnosis of AD is frequently subjective and is determined by physicians (generally neurologists, geriatricians, and psychiatrists) depending on their experience. Diagnosing AD requires both medical history and mental status testing. Having trouble with memory does not mean you have AD. AD has no current cure, but treatments for symptoms are available and research continues. In this study, we investigated the potential of infrared microscopy to differentiate between AD patients and controls, using Fourier transform infrared (FTIR) spectroscopy of isolated blood components. FTIR is known as a quick, safe, and minimally invasive method to investigate biological samples. For this goal, we measured infrared spectra from white blood cells (WBCs) and plasma taken from AD patients and controls, with the consent of the patients or their guardians. Applying multivariate analysis, principal component analysis (PCA) followed by linear discriminant analysis (LDA), it was possible to differentiate among the different types of mild, moderate, and severe AD, and the controls, with 85% accuracy when using the WBC spectra and about 77% when using the plasma spectra. When only the moderate and severe stages were included, an 83% accuracy was obtained using the WBC spectra and about 89% when using the plasma spectra.

SIproc: an open-source biomedical data processing platform for large hyperspectral images

There has recently been significant interest within the vibrational spectroscopy community to apply quantitative spectroscopic imaging techniques to histology and clinical diagnosis. However, many of the proposed methods require collecting spectroscopic images that have a similar region size and resolution to the corresponding histological images. Since spectroscopic images contain significantly more spectral samples than traditional histology, the resulting data sets can approach hundreds of gigabytes to terabytes in size. This makes them difficult to store and process, and the tools available to researchers for handling large spectroscopic data sets are limited. Fundamental mathematical tools, such as MATLAB, Octave, and SciPy, are extremely powerful but require that the data be stored in fast memory. This memory limitation becomes impractical for even modestly sized histological images, which can be hundreds of gigabytes in size. In this paper, we propose an open-source toolkit designed to perform out-of-core processing of hyperspectral images. By taking advantage of graphical processing unit (GPU) computing combined with adaptive data streaming, our software alleviates common workstation memory limitations while achieving better performance than existing applications.

Mid-infrared spectral microimaging of inflammatory skin lesions

Skin is one of the most important organs of the human body because of its characteristics and functions. There are many alterations, either pathological or physiological, that can disturb its functioning. However, at present all methods used to investigate skin diseases, non-invasive or invasive, are based on clinical examinations by physicians. Thus, diagnosis, prognosis and therapeutic management rely on the expertise of the practitioner, the quality of the method and the accessibility of distinctive morphological characteristics of each lesion. To overcome the high sensitivity of these parameters, techniques based on more objective criteria must be explored. Vibrational spectroscopy has become as a key technique for tissue analysis in the biomedical research field. Based on a non-destructive light/matter interaction, this tool provides information about specific molecular structure and composition of the analyzed sample, thus relating to its precise physiopathological state and permitting to distinguish lesional from normal tissues. This label-free optical method can be performed directly on the paraffin-embedded tissue sections without chemical dewaxing. In this study, the potential of the infrared microspectroscopy, combined with data classification methods was demonstrated, to characterize at the tissular level different types of inflammatory skin lesions, and this independently from conventional histopathology.

Differential Diagnosis of the Etiologies of Bacterial and Viral Infections Using Infrared Microscopy of Peripheral Human Blood Samples and Multivariate Analysis

Human viral and bacterial infections are responsible for a variety of diseases that are still the main causes of death and economic burden for society across the globe. Despite the different responses of the immune system to these infections, some of them have similar symptoms, such as fever, sneezing, inflammation, vomiting, diarrhea, and fatigue. Thus, physicians usually encounter difficulties in distinguishing between viral and bacterial infections on the basis of these symptoms. Rapid identification of the etiology of infection is highly important for effective treatment and can save lives in some cases. The current methods used for the identification of the nature of the infection are mainly based on growing the infective agent in culture, which is a time-consuming (over 24 h) and usually expensive process. The main objective of this study was to evaluate the potential of the mid-infrared spectroscopic method for rapid and reliable identification of bacterial and viral infections based on simple peripheral blood samples. For this purpose, white blood cells (WBCs) and plasma were isolated from the peripheral blood samples of patients with confirmed viral or bacterial infections. The obtained spectra were analyzed by multivariate analysis: principle component analysis (PCA) followed by linear discriminant analysis (LDA), to identify the infectious agent type as bacterial or viral in a time span of about 1 h after the collection of the blood sample. Our preliminary results showed that it is possible to determine the infectious agent with high success rates of 82% for sensitivity and 80% for specificity, based on the WBC data.

Infrared micro-spectroscopy of human tissue: principles and future promises

This article summarizes the methods employed, and the progress achieved over the past two decades in applying vibrational (Raman and IR) micro-spectroscopy to problems of medical diagnostics and cellular biology. During this time, several research groups have verified the enormous information contained in vibrational spectra; in fact, information on protein, lipid and metabolic composition of cells and tissues can be deduced by decoding the observed vibrational spectra. This decoding process is aided by the availability of computer workstations and advanced algorithms for data analysis. Furthermore, commercial instrumentation for the fast collection of both Raman and infrared micro-spectral data has enabled the collection of images of cells and tissues based solely on vibrational spectroscopic data. The progress in the field has been manifested by a steady increase in the number and quality of publications submitted by established and new research groups in vibrational spectroscopy in the biological and biomedical arenas.

Roadmap on optical sensors

Sensors are devices or systems able to detect, measure and convert magnitudes from any domain to an electrical one. Using light as a probe for optical sensing is one of the most efficient approaches for this purpose. The history of optical sensing using some methods based on absorbance, emissive and florescence properties date back to the 16th century. The field of optical sensors evolved during the following centuries, but it did not achieve maturity until the demonstration of the first laser in 1960. The unique properties of laser light become particularly important in the case of laser-based sensors, whose operation is entirely based upon the direct detection of laser light itself, without relying on any additional mediating device. However, compared with freely propagating light beams, artificially engineered optical fields are in increasing demand for probing samples with very small sizes and/or weak light-matter interaction. Optical fiber sensors constitute a subarea of optical sensors in which fiber technologies are employed. Different types of specialty and photonic crystal fibers provide improved performance and novel sensing concepts. Actually, structurization with wavelength or subwavelength feature size appears as the most efficient way to enhance sensor sensitivity and its detection limit. This leads to the area of micro- and nano-engineered optical sensors. It is expected that the combination of better fabrication techniques and new physical effects may open new and fascinating opportunities in this area. This roadmap on optical sensors addresses different technologies and application areas of the field. Fourteen contributions authored by experts from both industry and academia provide insights into the current state-of-the-art and the challenges faced by researchers currently. Two sections of this paper provide an overview of laser-based and frequency comb-based sensors. Three sections address the area of optical fiber sensors, encompassing both conventional, specialty and photonic crystal fibers. Several other sections are dedicated to micro- and nano-engineered sensors, including whispering-gallery mode and plasmonic sensors. The uses of optical sensors in chemical, biological and biomedical areas are described in other sections. Different approaches required to satisfy applications at visible, infrared and THz spectral regions are also discussed. Advances in science and technology required to meet challenges faced in each of these areas are addressed, together with suggestions on how the field could evolve in the near future.

Label-Free Molecular Imaging of Biological Cells and Tissues by Linear and Nonlinear Raman Spectroscopic Approaches

Raman spectroscopy is an emerging technique in bioanalysis and imaging of biomaterials owing to its unique capability of generating spectroscopic fingerprints. Imaging cells and tissues by Raman microspectroscopy represents a nondestructive and label-free approach. All components of cells or tissues contribute to the Raman signals, giving rise to complex spectral signatures. Resonance Raman scattering and surface-enhanced Raman scattering can be used to enhance the signals and reduce the spectral complexity. Raman-active labels can be introduced to increase specificity and multimodality. In addition, nonlinear coherent Raman scattering methods offer higher sensitivities, which enable the rapid imaging of larger sampling areas. Finally, fiber-based imaging techniques pave the way towards in vivo applications of Raman spectroscopy. This Review summarizes the basic principles behind medical Raman imaging and its progress since 2012.

Fundamental developments in infrared spectroscopic imaging for biomedical applications

Infrared chemical imaging is a rapidly emerging field with new advances in instrumentation, data acquisition and data analysis. These developments have had significant impact in biomedical applications and numerous studies have now shown that this technology offers great promise for the improved diagnosis of the diseased state. Relying on purely biochemical signatures rather than contrast from exogenous dyes and stains, infrared chemical imaging has the potential to revolutionise histopathology for improved disease diagnosis. In this review we discuss the recent advances in infrared spectroscopic imaging specifically related to spectral histopathology (SHP) and consider the current state of the field. Finally we consider the practical application of SHP for disease diagnosis and consider potential barriers to clinical translation highlighting current directions and the future outlook.

Classification of malignant and benign tumors of the lung by infrared spectral histopathology (SHP)

We report results of a study utilizing a novel tissue classification method, based on label-free spectral techniques, for the classification of lung cancer histopathological samples on a tissue microarray. The spectral diagnostic method allows reproducible and objective classification of unstained tissue sections. This is accomplished by acquiring infrared data sets containing thousands of spectra, each collected from tissue pixels ∼6 μm on edge; these pixel spectra contain an encoded snapshot of the entire biochemical composition of the pixel area. The hyperspectral data sets are subsequently decoded by methods of multivariate analysis that reveal changes in the biochemical composition between tissue types, and between various stages and states of disease. In this study, a detailed comparison between classical and spectral histopathology is presented, suggesting that spectral histopathology can achieve levels of diagnostic accuracy that is comparable to that of multipanel immunohistochemistry.