Detection of pancreatic cancer by convolutional-neural-network-assisted spontaneous Raman spectroscopy with critical feature visualization

A double-branch convolutional neural network model for species identification based on multi-modal data

For species identification analysis, methods based on deep learning are becoming prevalent due to their data-driven and task-oriented nature. The most commonly used convolutional neural network (CNN) model has been well applied in Raman spectra recognition. However, when faced with similar molecules or functional groups, the features of overlapping peaks and weak peaks may not be fully extracted using the CNN model, which can potentially hinder accurate species identification. Based on these practical challenges, the fusion of multi-modal data can effectively meet the comprehensive and accurate analysis of actual samples when compared with single-modal data. In this study, we propose a double-branch CNN model by integrating Raman and image multi-modal data, named SI-DBNet. In addition, we have developed a one-dimensional convolutional neural network combining dilated convolutions and efficient channel attention mechanisms for spectral branching. The effectiveness of the model has been demonstrated using the Grad-CAM method to visualize the key regions concerned by the model. When compared to single-modal and multi-modal classification methods, our SI-DBNet model achieved superior performance with a classification accuracy of 98.8%. The proposed method provided a new reference for species identification based on multi-modal data fusion.

Self-immunological disease aid diagnosis with ConvSANet and Eu-clidean distance

BACKGROUND

Ankylosing spondylitis (AS), Osteoarthritis (OA), and Sjögren's syndrome (SS) are three prevalent autoimmune diseases. If left untreated, which can lead to severe joint damage and greatly limit mobility. Once the disease worsens, patients may face the risk of long-term disability, and in severe cases, even life-threatening consequences.

RESULT

In this study, the Raman spectral data of AS, OA, and SS are analyzed to auxiliary disease diagnosis. For the first time, the Euclidean distance(ED) upscaling technique was used for the conversation from one-dimensional(1D) disease spectral data to two-dimensional(2D) spectral images. A dual-attention mechanism network was then constructed to analyze these two-dimensional spectral maps for disease diagnosis. The results demonstrate that the dual-attention mechanism network achieves a diagnostic accuracy of 100 % when analyzing 2D ED spectrograms. Furthermore, a comparison and analysis with s-transforms(ST), short-time fourier transforms(STFT), recurrence maps(RP), markov transform field(MTF), and Gramian angle fields(GAF) highlight the significant advantage of the proposed method, as it significantly shortens the conversion time while supporting disease-assisted diagnosis. Mutual information(MI) was utilized for the first time to validate the 2D Raman spectrograms generated, including ED, ST, STFT, RP, MTF, and GAF spectrograms. This allowed for evaluation of the similarity between the original 1D spectral data and the generated 2D spectrograms.

SIGNIFICANT

The results indicate that utilizing ED to transform 1D spectral data into 2D images, coupled with the application of convolutional neural network(CNN) for analyzing 2D ED Raman spectrograms, holds great promise as a valuable tool in assisting disease diagnosis. The research demonstrated that the 2D spectrogram created with ED closely resembles the original 1D spectral data. This indicates that ED effectively captures key features and important information from the original data, providing a strong descript.

Quantitative analysis of the quality constituents of Lonicera japonica Thunberg based on Raman spectroscopy

Quantitative analysis of the quality constituents of Lonicera japonica (Jinyinhua [JYH]) using a feasible method provides important information on its evaluation and applications. Limitations of sample pretreatment, experimental site, and analysis time should be considered when identifying new methods. In response to these considerations, Raman spectroscopy combined with deep learning was used to establish a quantitative analysis model to determine the quality of JYH. Chlorogenic acid and total flavonoids were identified as analysis targets via network pharmacology. High performance liquid chromatograph and ultraviolet spectroscopy were used to construct standard curves for quantitative analysis. Raman spectra of JYH extracts (1200) were collected. Subsequently, models were built using partial least squares regression, Support Vector Machine, Back Propagation Neural Network, and One-dimensional Convolutional Neural Network (1D-CNN). Among these, the 1D-CNN model showed superior prediction capability and had higher accuracy (R2 = 0.971), and lower root mean square error, indicating its suitability for rapid quantitative analysis.

Detection of lung cancer through SERS analysis of serum

Early detection and postoperative assessment are crucial for improving overall survival among lung cancer patients. Here, we report a non-invasive technique that integrates Raman spectroscopy with machine learning for the detection of lung cancer. The study encompassed 88 postoperative lung cancer patients, 73 non-surgical lung cancer patients, and 68 healthy subjects. The primary aim was to explore variations in serum metabolism across these cohorts. Comparative analysis of average Raman spectra was conducted, while principal component analysis was employed for data visualization. Subsequently, the augmented dataset was used to train convolutional neural networks (CNN) and Resnet models, leading to the development of a diagnostic framework. The CNN model exhibited superior performance, as verified by the receiver operating characteristic curve. Notably, postoperative patients demonstrated an increased likelihood of recurrence, emphasizing the crucial need for continuous postoperative monitoring. In summary, the integration of Raman spectroscopy with CNN-based classification shows potential for early detection and postoperative assessment of lung cancer.

Serum analysis based on SERS combined with 2D convolutional neural network and Gramian angular field for breast cancer screening

Breast cancer is a significant cause of death among women worldwide. It is crucial to quickly and accurately diagnose breast cancer in order to reduce mortality rates. While traditional diagnostic techniques for medical imaging and pathology samples have been commonly used in breast cancer screening, they still have certain limitations. Surface-enhanced Raman spectroscopy (SERS) is a fast, highly sensitive and user-friendly method that is often combined with deep learning techniques like convolutional neural networks. This combination helps identify unique molecular spectral features, also known as "fingerprint", in biological samples such as serum. Ultimately, this approach is able to accurately screen for cancer. The Gramian angular field (GAF) algorithm can convert one-dimensional (1D) time series into two-dimensional (2D) images. These images can be used for data visualization, pattern recognition and machine learning tasks. In this study, 640 serum SERS from breast cancer patients and healthy volunteers were converted into 2D spectral images by Gramian angular field (GAF) technique. These images were then used to train and test a two-dimensional convolutional neural network-GAF (2D-CNN-GAF) model for breast cancer classification. We compared the performance of the 2D-CNN-GAF model with other methods, including one-dimensional convolutional neural network (1D-CNN), support vector machine (SVM), K-nearest neighbor (KNN) and principal component analysis-linear discriminant analysis (PCA-LDA), using various evaluation metrics such as accuracy, precision, sensitivity, F1-score, receiver operating characteristic (ROC) curve and area under curve (AUC) value. The results showed that the 2D-CNN model outperformed the traditional models, achieving an AUC value of 0.9884, an accuracy of 98.13%, sensitivity of 98.65% and specificity of 97.67% for breast cancer classification. In this study, we used conventional nano-silver sol as the SERS-enhanced substrate and a portable laser Raman spectrometer to obtain the serum SERS data. The 2D-CNN-GAF model demonstrated accurate and automatic classification of breast cancer patients and healthy volunteers. The method does not require augmentation and preprocessing of spectral data, simplifying the processing steps of spectral data. This method has great potential for accurate breast cancer screening and also provides a useful reference in more types of cancer classification and automatic screening.

A novel method for detection of pancreatic Ductal Adenocarcinoma using explainable machine learning

BACKGROUND AND OBJECTIVE

Pancreatic Ductal Adenocarcinoma (PDAC) is a form of pancreatic cancer that is one of the primary causes of cancer-related deaths globally, with less than 10 % of the five years survival rate. The prognosis of pancreatic cancer has remained poor in the last four decades, mainly due to the lack of early diagnostic mechanisms. This study proposes a novel method for detecting PDAC using explainable and supervised machine learning from Raman spectroscopic signals.

METHODS

An insightful feature set consisting of statistical, peak, and extended empirical mode decomposition features is selected using the support vector machine recursive feature elimination method integrated with a correlation bias reduction. Explicable features successfully identified mutations in Kirsten rat sarcoma viral oncogene homolog (KRAS) and tumor suppressor protein53 (TP53) in the fingerprint region for the first time in the literature. PDAC and normal pancreas are classified using K-nearest neighbor, linear discriminant analysis, and support vector machine classifiers.

RESULTS

This study achieved a classification accuracy of 98.5% using a nonlinear support vector machine. Our proposed method reduced test time by 28.5 % and saved 85.6 % memory utilization, which reduces complexity significantly and is more accurate than the state-of-the-art method. The generalization of the proposed method is assessed by fifteen-fold cross-validation, and its performance is evaluated using accuracy, specificity, sensitivity, and receiver operating characteristic curves.

CONCLUSIONS

In this study, we proposed a method to detect and define the fingerprint region for PDAC using explainable machine learning. This simple, accurate, and efficient method for PDAC detection in mice could be generalized to examine human pancreatic cancer and provide a basis for precise chemotherapy for early cancer treatment.

RaT: Raman Transformer for highly accurate melanoma detection with critical features visualization

Melanoma is an important cause of death from skin cancer. Early and accurate diagnosis can effectively reduce mortality. But the current diagnosis relies on the experience of pathologists, increasing the rate of misdiagnosis. In this paper, Raman Transformer (RaT) model is proposed by combining Raman spectroscopy and a Transformer encoder to distinguish the Raman spectra of melanoma and normal tissue. To make the spectral data more suitable for the Transformer encoder, we split the Raman spectrum into segments and map them into block vectors, which are then input into the Transformer encoder and classified using the multi-head self-attention mechanism and the Multilayer Perceptron (MLP). The RaT model achieves 99.69% accuracy, 99.61% sensitivity, and 99.82% specificity, which is higher than the classical principal component analysis with the neural network (PCA + NNET) method. In addition, we visualize and explain the fingerprint peaks found by the RaT model and their corresponding biological information. Our proposed RaT model provides a novel and reliable method for processing Raman spectral data, which is expected to help distinguish melanoma from normal cells, diagnose other diseases, and save human lives.

Artificial Intelligence for Surface-Enhanced Raman Spectroscopy

Surface-enhanced Raman spectroscopy (SERS), well acknowledged as a fingerprinting and sensitive analytical technique, has exerted high applicational value in a broad range of fields including biomedicine, environmental protection, food safety among the others. In the endless pursuit of ever-sensitive, robust, and comprehensive sensing and imaging, advancements keep emerging in the whole pipeline of SERS, from the design of SERS substrates and reporter molecules, synthetic route planning, instrument refinement, to data preprocessing and analysis methods. Artificial intelligence (AI), which is created to imitate and eventually exceed human behaviors, has exhibited its power in learning high-level representations and recognizing complicated patterns with exceptional automaticity. Therefore, facing up with the intertwining influential factors and explosive data size, AI has been increasingly leveraged in all the above-mentioned aspects in SERS, presenting elite efficiency in accelerating systematic optimization and deepening understanding about the fundamental physics and spectral data, which far transcends human labors and conventional computations. In this review, the recent progresses in SERS are summarized through the integration of AI, and new insights of the challenges and perspectives are provided in aim to better gear SERS toward the fast track.

Unveiling breast cancer risk profiles: a survival clustering analysis empowered by an online web application

Aim: To develop a shiny app for doctors to investigate breast cancer treatments through a new approach by incorporating unsupervised clustering and survival information. Materials & methods: Analysis is based on the Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) dataset, which contains 1726 subjects and 22 variables. Cox regression was used to identify survival risk factors for K-means clustering. Logrank tests and C-statistics were compared across different cluster numbers and Kaplan-Meier plots were presented. Results & conclusion: Our study fills an existing void by introducing a unique combination of unsupervised learning techniques and survival information on the clinician side, demonstrating the potential of survival clustering as a valuable tool in uncovering hidden structures based on distinct risk profiles.

Combined analytical approach empowers precise spectroscopic interpretation of subcellular components of pancreatic cancer cells

The lack of specific and sensitive early diagnostic options for pancreatic cancer (PC) results in patients being largely diagnosed with late-stage disease, thus inoperable and burdened with high mortality. Molecular spectroscopic methodologies, such as Raman or infrared spectroscopies, show promise in becoming a leader in screening for early-stage cancer diseases, including PC. However, should such technology be introduced, the identification of differentiating spectral features between various cancer types is required. This would not be possible without the precise extraction of spectra without the contamination by necrosis, inflammation, desmoplasia, or extracellular fluids such as mucous that surround tumor cells. Moreover, an efficient methodology for their interpretation has not been well defined. In this study, we compared different methods of spectral analysis to find the best for investigating the biomolecular composition of PC cells cytoplasm and nuclei separately. Sixteen PC tissue samples of main PC subtypes (ductal adenocarcinoma, intraductal papillary mucinous carcinoma, and ampulla of Vater carcinoma) were collected with Raman hyperspectral mapping, resulting in 191,355 Raman spectra and analyzed with comparative methodologies, specifically, hierarchical cluster analysis, non-negative matrix factorization, T-distributed stochastic neighbor embedding, principal components analysis (PCA), and convolutional neural networks (CNN). As a result, we propose an innovative approach to spectra classification by CNN, combined with PCA for molecular characterization. The CNN-based spectra classification achieved over 98% successful validation rate. Subsequent analyses of spectral features revealed differences among PC subtypes and between the cytoplasm and nuclei of their cells. Our study establishes an optimal methodology for cancer tissue spectral data classification and interpretation that allows precise and cognitive studies of cancer cells and their subcellular components, without mixing the results with cancer-surrounding tissue. As a proof of concept, we describe findings that add to the spectroscopic understanding of PC.

Diagnoses in multiple types of cancer based on serum Raman spectroscopy combined with a convolutional neural network: Gastric cancer, colon cancer, rectal cancer, lung cancer

Cancer is one of the major diseases that seriously threaten human health. Timely screening is beneficial to the cure of cancer. There are some shortcomings in current diagnosis methods, so it is very important to find a low-cost, fast, and nondestructive cancer screening technology. In this study, we demonstrated that serum Raman spectroscopy combined with a convolutional neural network model can be used for the diagnosis of four types of cancer including gastric cancer, colon cancer, rectal cancer, and lung cancer. Raman spectra database containing four types of cancer and healthy controls was established and a one-dimensional convolutional neural network (1D-CNN) was constructed. The classification accuracy of the Raman spectra combined with the 1D-CNN model was 94.5%. A convolutional neural network (CNN) is regarded as a black box, and the learning mechanism of the model is not clear. Therefore, we tried to visualize the CNN features of each convolutional layer in the diagnosis of rectal cancer. Overall, Raman spectroscopy combined with the CNN model is an effective tool that can be used to distinguish different cancer from healthy controls.

Practices, Potential, and Perspectives for Detecting Predisease Using Raman Spectroscopy

Raman spectroscopy shows great potential for practical clinical applications. By analyzing the structure and composition of molecules through real-time, non-destructive measurements of the scattered light from living cells and tissues, it offers valuable insights. The Raman spectral data directly link to the molecular composition of the cells and tissues and provides a "molecular fingerprint" for various disease states. This review focuses on the practical and clinical applications of Raman spectroscopy, especially in the early detection of human diseases. Identifying predisease, which marks the transition from a healthy to a disease state, is crucial for effective interventions to prevent disease onset. Raman spectroscopy can reveal biological processes occurring during the transition states and may eventually detect the molecular dynamics in predisease conditions.

Original and liposome-modified indocyanine green-assisted fluorescence study with animal models

Medical diagnosis heavily relies on the use of bio-imaging techniques. One such technique is the use of ICG-based biological sensors for fluorescence imaging. In this study, we aimed to improve the fluorescence signals of ICG-based biological sensors by incorporating liposome-modified ICG. The results from dynamic light scattering and transmission electron microscopy showed that MLM-ICG was successfully fabricated with a liposome diameter of 100-300 nm. Fluorescence spectroscopy showed that MLM-ICG had the best properties among the three samples (Blank ICG, LM-ICG, and MLM-ICG), as samples immersed in MLM-ICG solution achieved the highest fluorescence intensity. The NIR camera imaging also showed a similar result. For the rat model, the best period for fluorescence tests was between 10 min and 4 h, where most organs reached their maximum fluorescence intensity except for the liver, which continued to rise. After 24 h, ICG was excreted from the rat's body. The study also analyzed the spectra properties of different rat organs, including peak intensity, peak wavelength, and FWHM. In conclusion, the use of liposome-modified ICG provides a safe and optimized optical agent, which is more stable and efficient than non-modified ICG. Incorporating liposome-modified ICG in fluorescence spectroscopy could be an effective way to develop novel biosensors for disease diagnosis.

A multi-view co-training network for semi-supervised medical image-based prognostic prediction

Prognostic prediction has long been a hotspot in disease analysis and management, and the development of image-based prognostic prediction models has significant clinical implications for current personalized treatment strategies. The main challenge in prognostic prediction is to model a regression problem based on censored observations, and semi-supervised learning has the potential to play an important role in improving the utilization efficiency of censored data. However, there are yet few effective semi-supervised paradigms to be applied. In this paper, we propose a semi-supervised co-training deep neural network incorporating a support vector regression layer for survival time estimation (Co-DeepSVS) that improves the efficiency in utilizing censored data for prognostic prediction. First, we introduce a support vector regression layer in deep neural networks to deal with censored data and directly predict survival time, and more importantly to calculate the labeling confidence of each case. Then, we apply a semi-supervised multi-view co-training framework to achieve accurate prognostic prediction, where labeling confidence estimation with prior knowledge of pseudo time is conducted for each view. Experimental results demonstrate that the proposed Co-DeepSVS has a promising prognostic ability and surpasses most widely used methods on a multi-phase CT dataset. Besides, the introduction of SVR layer makes the model more robust in the presence of follow-up bias.

A classification model for detection of ductal carcinoma in situ by Fourier transform infrared spectroscopy based on deep structured semantic model

At present, deep learning is widely used in spectral data processing. Deep learning requires a large amount of data for training, while the collection of biological serum spectra is limited by sample numbers and labor costs, so it is impractical to obtain a large amount of serum spectral data for disease detection. In this study, we propose a spectral classification model based on the deep structured semantic model (DSSM) and successfully apply it to Fourier Transform Infrared (FT-IR) spectroscopy for ductal carcinoma in situ (DCIS) detection. Compared with the traditional deep learning model, we match the spectral data into positive and negative pairs according to whether the spectra are from the same category. The DSSM structure is constructed by extracting features according to the spectral similarity of spectra pairs. This new construction model increases the data amount used for model training and reduces the dimension of spectral data. Firstly, the FT-IR spectra are paired. The spectra pairs are labeled as positive pairs if they come from the same category, and the spectra pairs are labeled as negative pairs if they come from different categories. Secondly, two spectra in each spectra pair are put into two deep neural networks of the DSSM structure separately. Then the spectral similarity between the output feature maps of two deep neural networks is calculated. The DSSM structure is trained by maximizing the conditional likelihood of the spectra pairs from the same category. Thirdly, after the training of DSSM is done, the training set and testing set are input into two deep neural networks separately. The output feature maps of the training set are put into the reference library. Lastly, the k-nearest neighbor (KNN) model is used for classification according to Euclidean distances between the output feature map of each unknown sample to the reference library. The category of the unknown sample is judged according to the categories of k nearest samples. We also use principal component analysis (PCA) to reduce dimension for comparison. The accuracies of the KNN model, principal component analysis-k nearest neighbor (PCA-KNN) model, and deep structured semantic model-k nearest neighbor (DSSM-KNN) model are 78.8%, 72.7%, and 97.0%, which proves that our proposed model has higher accuracy.

Industrial wastewater source tracing: The initiative of SERS spectral signature aided by a one-dimensional convolutional neural network

The spectral fingerprint is a significant concept in nontarget screening of environmental samples to direct identification efforts to relevant and important features. Surface-enhanced Raman scattering (SERS) has long been recognized as an optical method that can provide fingerprint-like chemical information at the single-molecule level. Here, the advanced one-dimensional convolutional neural network (1D-CNN) approach was applied to accurately identify the SERS spectral signature of industrial wastewaters for source tracing. A total of 66,000 SERS spectra were acquired from wastewaters of 22 factories across 10 industrial categories at three excitation wavelengths after data augmentation. The dataset was used to train a 1D-CNN model consisting of three convolutional layers to achieve adequate feature extraction of SERS spectra. As a proof-of-concept, multimixed wastewater samples were used to simulate practical pollution scenarios and evaluate the application potential of the model. The SERS-1D-CNN platform can identify the amount and factory information of wastewaters in multimixed samples, which achieves a recognition accuracy rate of 97.33%. The results suggest that even in a complex and unknown water environment, the 1D-CNN model can accurately identify industrial wastewaters in precollected datasets, exhibiting excellent potential in pollution source tracing.

Classification of pathogenic bacteria by Raman spectroscopy combined with variational auto-encoder and deep learning

Rapid and early identification of pathogens is critical to guide antibiotic therapy. Raman spectroscopy as a noninvasive diagnostic technique provides rapid and accurate detection of pathogens. Raman spectrum of single cells serves as the "fingerprint" of the cell, revealing its metabolic characteristics. Rapid identification of pathogens can be achieved by combining Raman spectroscopy and deep learning. Traditional classification techniques frequently require lots of data for training, which is time costing to collect Raman spectra. For trace samples and strains that are difficult to culture, it is difficult to provide an accurate classification model. In order to reduce the number of samples collected and improve the accuracy of the classification model, a new pathogen detection method integrating Raman spectroscopy, variational auto-encoder (VAE), and long short-term memory network (LSTM) is proposed in this paper. We collect the Raman signals of pathogens and input them to VAE for training. VAE will generate a large number of Raman spectral data that cannot be distinguished from the real spectrum, and the signal-to-noise ratio is higher than that of the real spectrum. These spectra are input into the LSTM together with the real spectrum for training, and a good classification model is obtained. The results of the experiments reveal that this method not only improves the average accuracy of pathogen classification to 96.9% but also reduces the number of Raman spectra collected from 1000 to 200. With this technology, the number of Raman spectra collected can be greatly reduced, so that strains that are difficult to culture or trace can be rapidly identified.

Accurate identification of living Bacillus spores using laser tweezers Raman spectroscopy and deep learning

Accurately, rapidly, and noninvasively identifying Bacillus spores can greatly contribute to controlling a plenty of infectious diseases. Laser tweezers Raman spectroscopy (LTRS) has confirmed to be a powerful tool for studying Bacillus spores at a single cell level. In this study, we constructed a single-cell Raman spectra dataset of living Bacillus spores and utilized deep learning approach to accurately, nondestructively identify Bacillus spores. The trained convolutional neural network (CNN) could efficiently extract tiny Raman spectra features of five spore species, and provide a prediction accuracy of specie identification as high as 100 %. Moreover, the spectral feature differences in three Raman bands at 660, 826, and 1017 cm^-1 were confirmed to mostly contribute to producing such high prediction accuracy. In addition, optimal CNN model was employed to monitor and identify sporulation process at different metabolic phases in one growth cycle. The obtained average prediction accuracy of metabolic phase identification was approximately 88 %. It can be foreseen that, LTRS combined with CNN approach have great potential for accurately identifying spore species and metabolic phases at a single cell level, and can be gradually extended to perform identification for many unculturable bacteria growing in soil, water, and food.

Variabilities in global DNA methylation and β-sheet richness establish spectroscopic landscapes among subtypes of pancreatic cancer

PURPOSE

Knowledge about pancreatic cancer (PC) biology has been growing rapidly in recent decades. Nevertheless, the survival of PC patients has not greatly improved. The development of a novel methodology suitable for deep investigation of the nature of PC tumors is of great importance. Molecular imaging techniques, such as Fourier transform infrared (FTIR) spectroscopy and Raman hyperspectral mapping (RHM) combined with advanced multivariate data analysis, were useful in studying the biochemical composition of PC tissue.

METHODS

Here, we evaluated the potential of molecular imaging in differentiating three groups of PC tumors, which originate from different precursor lesions. Specifically, we comprehensively investigated adenocarcinomas (ACs): conventional ductal AC, intraductal papillary mucinous carcinoma, and ampulla of Vater AC. FTIR microspectroscopy and RHM maps of 24 PC tissue slides were obtained, and comprehensive advanced statistical analyses, such as hierarchical clustering and nonnegative matrix factorization, were performed on a total of 211,355 Raman spectra. Additionally, we employed deep learning technology for the same task of PC subtyping to enable automation. The so-called convolutional neural network (CNN) was trained to recognize spectra specific to each PC group and then employed to generate CNN-prediction-based tissue maps. To identify the DNA methylation spectral markers, we used differently methylated, isolated DNA and compared the observed spectral differences with the results obtained from cellular nuclei regions of PC tissues.

RESULTS

The results showed significant differences among cancer tissues of the studied PC groups. The main findings are the varying content of β-sheet-rich proteins within the PC cells and alterations in the relative DNA methylation level. Our CNN model efficiently differentiated PC groups with 94% accuracy. The usage of CNN in the classification task did not require Raman spectral data preprocessing and eliminated the need for extensive knowledge of statistical methodologies.

CONCLUSIONS

Molecular spectroscopy combined with CNN technology is a powerful tool for PC detection and subtyping. The molecular fingerprint of DNA methylation and β-sheet cytoplasmic proteins established by our results is different for the main PC groups and allowed the subtyping of pancreatic tumors, which can improve patient management and increase their survival. Our observations are of key importance in understanding the variability of PC and allow translation of the methodology into clinical practice by utilizing liquid biopsy testing.

Investigation of probability maps in deep-learning-based brain ventricle parcellation

Normal Pressure Hydrocephalus (NPH) is a brain disorder associated with ventriculomegaly. Accurate segmentation of the ventricle system into its sub-compartments from magnetic resonance images (MRIs) could help evaluate NPH patients for surgical intervention. In this paper, we modify a 3D U-net utilizing probability maps to perform accurate ventricle parcellation, even with grossly enlarged ventricles and post-surgery shunt artifacts, from MRIs. Our method achieves a mean dice similarity coefficient (DSC) on whole ventricles for healthy controls of 0.864 ± 0.047 and 0.961 ± 0.024 for NPH patients. Furthermore, with the benefit of probability maps, the proposed method provides superior performance on MRI with grossly enlarged ventricles (mean DSC value of 0.965 ± 0.027) or post-surgery shunt artifacts (mean DSC value of 0.964 ± 0.031). Results indicate that our method provides a high robust parcellation tool on the ventricular systems which is comparable to other state-of-the-art methods.

Raman spectroscopy and convolutional neural networks for monitoring biochemical radiation response in breast tumour xenografts

Tumour cells exhibit altered metabolic pathways that lead to radiation resistance and disease progression. Raman spectroscopy (RS) is a label-free optical modality that can monitor post-irradiation biomolecular signatures in tumour cells and tissues. Convolutional Neural Networks (CNN) perform automated feature extraction directly from data, with classification accuracy exceeding that of traditional machine learning, in cases where data is abundant and feature extraction is challenging. We are interested in developing a CNN-based predictive model to characterize clinical tumour response to radiation therapy based on their degree of radiosensitivity or radioresistance. In this work, a CNN architecture is built for identifying post-irradiation spectral changes in Raman spectra of tumour tissue. The model was trained to classify irradiated versus non-irradiated tissue using Raman spectra of breast tumour xenografts. The CNN effectively classified the tissue spectra, with accuracies exceeding 92.1% for data collected 3 days post-irradiation, and 85.0% at day 1 post-irradiation. Furthermore, the CNN was evaluated using a leave-one-out- (mouse, section or Raman map) validation approach to investigate its generalization to new test subjects. The CNN retained good predictive accuracy (average accuracies 83.7%, 91.4%, and 92.7%, respectively) when little to no information for a specific subject was given during training. Finally, the classification performance of the CNN was compared to that of a previously developed model based on group and basis restricted non-negative matrix factorization and random forest (GBR-NMF-RF) classification. We found that CNN yielded higher classification accuracy, sensitivity, and specificity in mice assessed 3 days post-irradiation, as compared with the GBR-NMF-RF approach. Overall, the CNN can detect biochemical spectral changes in tumour tissue at an early time point following irradiation, without the need for previous manual feature extraction. This study lays the foundation for developing a predictive framework for patient radiation response monitoring.

Lock-in Raman difference spectroscopy

Shifted Excitation Raman Difference Spectroscopy (SERDS) is a non-destructive chemical analysis method capable of removing the fluorescence background and other disturbances from the Raman spectrum, thanks to the independence of the fluorescence with respect to the small difference in excitation wavelength. The spectrum difference is computed in a post-processing step. Here, we demonstrate the use of a lock-in camera to obtain an on-line analog SERDS spectra allowing longer exposure times and no saturation, leading to an improved Signal-to-Noise Ratio (SNR) and reduced data storage. Two configurations are presented: the first one uses a single laser and can remove excitation-independent disturbances, such as ambient light; the second employs two-wavelength shifted sources and removes fluorescence background similarly to SERDS. In both cases, we experimentally extrapolate the expected SNR improvement.

Emerging Applications of Deep Learning in Bone Tumors: Current Advances and Challenges

Deep learning is a subfield of state-of-the-art artificial intelligence (AI) technology, and multiple deep learning-based AI models have been applied to musculoskeletal diseases. Deep learning has shown the capability to assist clinical diagnosis and prognosis prediction in a spectrum of musculoskeletal disorders, including fracture detection, cartilage and spinal lesions identification, and osteoarthritis severity assessment. Meanwhile, deep learning has also been extensively explored in diverse tumors such as prostate, breast, and lung cancers. Recently, the application of deep learning emerges in bone tumors. A growing number of deep learning models have demonstrated good performance in detection, segmentation, classification, volume calculation, grading, and assessment of tumor necrosis rate in primary and metastatic bone tumors based on both radiological (such as X-ray, CT, MRI, SPECT) and pathological images, implicating a potential for diagnosis assistance and prognosis prediction of deep learning in bone tumors. In this review, we first summarized the workflows of deep learning methods in medical images and the current applications of deep learning-based AI for diagnosis and prognosis prediction in bone tumors. Moreover, the current challenges in the implementation of the deep learning method and future perspectives in this field were extensively discussed.

Multiple-Inputs Convolutional Neural Network for COVID-19 Classification and Critical Region Screening From Chest X-ray Radiographs: Model Development and Performance Evaluation

Background

The COVID-19 pandemic is becoming one of the largest, unprecedented health crises, and chest X-ray radiography (CXR) plays a vital role in diagnosing COVID-19. However, extracting and finding useful image features from CXRs demand a heavy workload for radiologists.

Objective

The aim of this study was to design a novel multiple-inputs (MI) convolutional neural network (CNN) for the classification of COVID-19 and extraction of critical regions from CXRs. We also investigated the effect of the number of inputs on the performance of our new MI-CNN model.

Methods

A total of 6205 CXR images (including 3021 COVID-19 CXRs and 3184 normal CXRs) were used to test our MI-CNN models. CXRs could be evenly segmented into different numbers (2, 4, and 16) of individual regions. Each region could individually serve as one of the MI-CNN inputs. The CNN features of these MI-CNN inputs would then be fused for COVID-19 classification. More importantly, the contributions of each CXR region could be evaluated through assessing the number of images that were accurately classified by their corresponding regions in the testing data sets.

Results

In both the whole-image and left- and right-lung region of interest (LR-ROI) data sets, MI-CNNs demonstrated good efficiency for COVID-19 classification. In particular, MI-CNNs with more inputs (2-, 4-, and 16-input MI-CNNs) had better efficiency in recognizing COVID-19 CXRs than the 1-input CNN. Compared to the whole-image data sets, the efficiency of LR-ROI data sets showed approximately 4% lower accuracy, sensitivity, specificity, and precision (over 91%). In considering the contributions of each region, one of the possible reasons for this reduced performance was that nonlung regions (eg, region 16) provided false-positive contributions to COVID-19 classification. The MI-CNN with the LR-ROI data set could provide a more accurate evaluation of the contribution of each region and COVID-19 classification. Additionally, the right-lung regions had higher contributions to the classification of COVID-19 CXRs, whereas the left-lung regions had higher contributions to identifying normal CXRs.

Conclusions

Overall, MI-CNNs could achieve higher accuracy with an increasing number of inputs (eg, 16-input MI-CNN). This approach could assist radiologists in identifying COVID-19 CXRs and in screening the critical regions related to COVID-19 classifications.