Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks

Prostate Cancer Risk Stratification via Nondestructive 3D Pathology with Deep Learning-Assisted Gland Analysis

Prostate cancer treatment planning is largely dependent upon examination of core-needle biopsies. The microscopic architecture of the prostate glands forms the basis for prognostic grading by pathologists. Interpretation of these convoluted three-dimensional (3D) glandular structures via visual inspection of a limited number of two-dimensional (2D) histology sections is often unreliable, which contributes to the under- and overtreatment of patients. To improve risk assessment and treatment decisions, we have developed a workflow for nondestructive 3D pathology and computational analysis of whole prostate biopsies labeled with a rapid and inexpensive fluorescent analogue of standard hematoxylin and eosin (H&E) staining. This analysis is based on interpretable glandular features and is facilitated by the development of image translation-assisted segmentation in 3D (ITAS3D). ITAS3D is a generalizable deep learning-based strategy that enables tissue microstructures to be volumetrically segmented in an annotation-free and objective (biomarker-based) manner without requiring immunolabeling. As a preliminary demonstration of the translational value of a computational 3D versus a computational 2D pathology approach, we imaged 300 ex vivo biopsies extracted from 50 archived radical prostatectomy specimens, of which, 118 biopsies contained cancer. The 3D glandular features in cancer biopsies were superior to corresponding 2D features for risk stratification of patients with low- to intermediate-risk prostate cancer based on their clinical biochemical recurrence outcomes. The results of this study support the use of computational 3D pathology for guiding the clinical management of prostate cancer. SIGNIFICANCE: An end-to-end pipeline for deep learning-assisted computational 3D histology analysis of whole prostate biopsies shows that nondestructive 3D pathology has the potential to enable superior prognostic stratification of patients with prostate cancer.

High-Throughput, Label-Free and Slide-Free Histological Imaging by Computational Microscopy and Unsupervised Learning

Rapid and high-resolution histological imaging with minimal tissue preparation has long been a challenging and yet captivating medical pursuit. Here, the authors propose a promising and transformative histological imaging method, termed computational high-throughput autofluorescence microscopy by pattern illumination (CHAMP). With the assistance of computational microscopy, CHAMP enables high-throughput and label-free imaging of thick and unprocessed tissues with large surface irregularity at an acquisition speed of 10 mm2 /10 s with 1.1-µm lateral resolution. Moreover, the CHAMP image can be transformed into a virtually stained histological image (Deep-CHAMP) through unsupervised learning within 15 s, where significant cellular features are quantitatively extracted with high accuracy. The versatility of CHAMP is experimentally demonstrated using mouse brain/kidney and human lung tissues prepared with various clinical protocols, which enables a rapid and accurate intraoperative/postoperative pathological examination without tissue processing or staining, demonstrating its great potential as an assistive imaging platform for surgeons and pathologists to provide optimal adjuvant treatment.

Differential characterization of lumbar spine associated tissue histology with nonlinear optical microscopy

Percutaneous endoscopic lumbar discectomy (PELD) is the major effective treatment for lumbar disc herniation, and rapid histological identification of dissected tissue is critical to guide the discectomy. In this work, we revealed the histological features of different types of peridural tissues of the lumbar spine by label-free multi-modal nonlinear optical microscopy. Stimulated Raman scattering (SRS) was used to extract lipid and protein distributions, while second harmonic generation (SHG) and two-photon excited fluorescence (TPEF) signals were applied to image the collagen and elastin fibers at the same time. Our results demonstrated that the nonlinear optical features of the dura and adjacent soft tissues were significantly different, showing the potentials of our method for intraoperative differentiation of these critical tissues and improving the surgical outcome of PELD.

Rapid intraoperative diagnosis of pediatric brain tumors using Raman spectroscopy: A machine learning approach

Background

Surgical resection is a mainstay in the treatment of pediatric brain tumors to achieve tissue diagnosis and tumor debulking. While maximal safe resection of tumors is desired, it can be challenging to differentiate normal brain from neoplastic tissue using only microscopic visualization, intraoperative navigation, and tactile feedback. Here, we investigate the potential for Raman spectroscopy (RS) to accurately diagnose pediatric brain tumors intraoperatively.

Methods

Using a rapid acquisition RS device, we intraoperatively imaged fresh ex vivo brain tissue samples from 29 pediatric patients at the Lucile Packard Children’s Hospital between October 2018 and March 2020 in a prospective fashion. Small tissue samples measuring 2-4 mm per dimension were obtained with each individual tissue sample undergoing multiple unique Raman spectra acquisitions. All tissue samples from which Raman spectra were acquired underwent individual histopathology review. A labeled dataset of 678 unique Raman spectra gathered from 160 samples was then used to develop a machine learning model capable of (1) differentiating normal brain from tumor tissue and (2) normal brain from low-grade glioma (LGG) tissue.

Results

Trained logistic regression model classifiers were developed using our labeled dataset. Model performance was evaluated using leave-one-patient-out cross-validation. The area under the curve (AUC) of the receiver-operating characteristic (ROC) curve for our tumor vs normal brain model was 0.94. The AUC of the ROC curve for LGG vs normal brain was 0.91.

Conclusions

Our work suggests that RS can be used to develop a machine learning-based classifier to differentiate tumor vs non-tumor tissue during resection of pediatric brain tumors.

Feasibility of using digital confocal microscopy for cytopathological examination in clinical practice

Optical imaging modalities are emerging as digital microscopy tools for tissue examination. The investigation of these techniques for potential applications in anatomic pathology practice has focused primarily on surgical pathology and has not included cytopathological specimens. We evaluated the feasibility of using digital confocal microscopy (CM) to examine cytopathological specimens. Smears and cell suspensions collected in RPMI solution were prepared from tissue scrapes obtained from surgical resections of breast, lung, liver, and kidney. Air-dried smears and cell pellets obtained from centrifugation of the cell suspensions were stained with 0.6 mM acridine orange and imaged with a CM platform. After completion of imaging, the smears were stained with Diff-Quik (DQ), and cell pellets were routinely processed, embedded in paraffin wax, cut, and stained with hematoxylin and eosin (H&E). We evaluated the mean time to acquire digital CM images; quality of images based on the extent of tissue recognition (0%, grade 0; 1-19%, grade 1; 20-50%, grade 2; >50%, grade 3); comparison of images with DQ- and H&E-stained specimens; and ability to make specific diagnoses. We imaged 91 smears and 52 cell pellets and acquired digital CM images within 2-3 min, with 92% and 88% of images, respectively, from smears and cell pellets showing grade 3 quality. On the basis of CM images, 8 smears (9%) and 7 cell pellets (14%) were categorized as benign, and 83 (91%) and 45 (88%), respectively, as malignant. Specific diagnoses were made by using digital CM images of smears and cell pellets that matched accurately with corresponding DQ- and H&E-stained preparations. The results of our first feasibility study clearly indicated the utility of CM as a next-generation digital microscopy tool for evaluating cytology specimens. Prospective clinical studies are warranted for validating our findings for potential incorporation into cytopathological clinical practice.

An Acidity-Initiated Self-Assembly/Disassembly Nanoprobe to Switch on Fluorescence for Tumor-Targeted Near-Infrared Imaging

The deep penetration, real-time monitoring ability, and high resolution of near-infrared (NIR) fluorescence imaging make it suitable for tumor diagnosis. However, the lack of specificity and selectivity restricts its further application. Here, for the first time, we applied a CBT-Cys click condensation reaction to synthesize an acidity-initiated molecular probe (AIM-Probe, Cys(StBu)-Lys(Cy 5.5)-EDA-PMA-CBT), which could self-assemble into nanoparticles (AIM-NP) with self-quenched fluorescence under glutathione (GSH) reduction. AIM-NP could accumulate in tumors after intravenous injection. Subsequently, the EDA-PMA part of AIM-Probe in AIM-NP is fractured by the unique subacid condition in the tumor microenvironment, and AIM-NP disassembles into a small AIM-cleaved molecule (PMA-CBT-Cys-Lys(Cy5.5)-EDA) along with fluorescence switching on. As a result, AIM-NP could switch on fluorescence at the tumor site, thereby achieving tumor-targeted imaging. To our knowledge, utilizing tumor acidity to initiate the disassembly of self-assembled nanoparticles through a CBT-Cys click condensation reaction has not been reported.

Artificial intelligence in clinical research of cancers

Several factors, including advances in computational algorithms, the availability of high-performance computing hardware, and the assembly of large community-based databases, have led to the extensive application of Artificial Intelligence (AI) in the biomedical domain for nearly 20 years. AI algorithms have attained expert-level performance in cancer research. However, only a few AI-based applications have been approved for use in the real world. Whether AI will eventually be capable of replacing medical experts has been a hot topic. In this article, we first summarize the cancer research status using AI in the past two decades, including the consensus on the procedure of AI based on an ideal paradigm and current efforts of the expertise and domain knowledge. Next, the available data of AI process in the biomedical domain are surveyed. Then, we review the methods and applications of AI in cancer clinical research categorized by the data types including radiographic imaging, cancer genome, medical records, drug information and biomedical literatures. At last, we discuss challenges in moving AI from theoretical research to real-world cancer research applications and the perspectives toward the future realization of AI participating cancer treatment.

Stimulated Raman histology in the neurosurgical workflow of a major European neurosurgical center - part A

Histopathological diagnosis is the current standard for the classification of brain and spine tumors. Raman spectroscopy has been reported to allow fast and easy intraoperative tissue analysis. Here, we report data on the intraoperative implementation of a stimulated Raman histology (SRH) as an innovative strategy offering intraoperative near real-time histopathological analysis. A total of 429 SRH images from 108 patients were generated and analyzed by using a Raman imaging system (Invenio Imaging Inc.). We aimed at establishing a dedicated workflow for SRH serving as an intraoperative diagnostic, research, and quality control tool in the neurosurgical operating room (OR). First experiences with this novel imaging modality were reported and analyzed suggesting process optimization regarding tissue collection, preparation, and imaging. The Raman imaging system was rapidly integrated into the surgical workflow of a large neurosurgical center. Within a few minutes of connecting the device, the first high-quality images could be acquired in a “plug-and-play” manner. We did not encounter relevant obstacles and the learning curve was steep. However, certain prerequisites regarding quality and acquisition of tissue samples, data processing and interpretation, and high throughput adaptions must be considered. Intraoperative SRH can easily be integrated into the workflow of neurosurgical tumor resection. Considering few process optimizations that can be implemented rapidly, high-quality images can be obtained near real time. Hence, we propose SRH as a complementary tool for the diagnosis of tumor entity, analysis of tumor infiltration zones, online quality and safety control and as a research tool in the neurosurgical OR.

Perspective review on applications of optics in skull base surgery

The use of optic technology in skull base surgeries has the potential to revolutionize the field of medicine, particularly neurosurgery and neurology. Here, we briefly present the past, present, and future of skull-base surgery, with an emphasis on the applications of optical topography techniques. We discuss optical topography techniques such as functional near-infrared spectroscopy, optical diffusion tomography, and optical topographical imaging. Optical topography techniques are particularly advantageous when combined with other imaging methods. For instance, optical topography can be combined with techniques such as functional magnetic resonance imaging (fMRI) to combine the temporal resolution of optical topography with the spatial resolution of fMRI. Multimodal approaches will be critical to advance brain-related research as well as medicine. Structured light imaging techniques are also writing the future of 3-dimensional imaging. In short, optical topography can allow for non-invasive, high-resolution imaging that will provide real-time visualizations of the brain that are ideal for neurosurgery. From the limitations of traditional skull base surgeries to the newest developments in optical neuroimaging, here we will discuss the potential applications of optics in skull base procedures.

Neuropathological interpretation of stimulated Raman histology images of brain and spine tumors: part B

Intraoperative histopathological examinations are routinely performed to provide neurosurgeons with information about the entity of tumor tissue. Here, we quantified the neuropathological interpretability of stimulated Raman histology (SRH) acquired using a Raman laser imaging system in a routine clinical setting without any specialized training or prior experience. Stimulated Raman scattering microscopy was performed on 117 samples of pathological tissue from 73 cases of brain and spine tumor surgeries. A board-certified neuropathologist — novice in the interpretation of SRH — assessed image quality by scoring subjective tumor infiltration and stated a diagnosis based on the SRH images. The diagnostic accuracy was determined by comparison to frozen hematoxylin–eosin (H&E)-stained sections and the ground truth defined as the definitive neuropathological diagnosis. The overall SRH imaging quality was rated high with the detection of tumor cells classified as inconclusive in only 4.2% of all images. The accuracy of neuropathological diagnosis based on SRH images was 87.7% and was non-inferior to the current standard of fast frozen H&E-stained sections (87.3 vs. 88.9%, p = 0.783). We found a substantial diagnostic correlation between SRH-based neuropathological diagnosis and H&E-stained frozen sections (κ = 0.8). The interpretability of intraoperative SRH imaging was demonstrated to be equivalent to the current standard method of H&E-stained frozen sections. Further research using this label-free innovative alternative vs. conventional staining is required to determine to which extent SRH-based intraoperative decision-making can be streamlined in order to facilitate the advancement of surgical neurooncology.

High-Throughput Molecular Imaging via Deep-Learning-Enabled Raman Spectroscopy

Raman spectroscopy enables nondestructive, label-free imaging with unprecedented molecular contrast, but is limited by slow data acquisition, largely preventing high-throughput imaging applications. Here, we present a comprehensive framework for higher-throughput molecular imaging via deep-learning-enabled Raman spectroscopy, termed DeepeR, trained on a large data set of hyperspectral Raman images, with over 1.5 million spectra (400 h of acquisition) in total. We first perform denoising and reconstruction of low signal-to-noise ratio Raman molecular signatures via deep learning, with a 10× improvement in the mean-squared error over common Raman filtering methods. Next, we develop a neural network for robust 2-4× spatial super-resolution of hyperspectral Raman images that preserve molecular cellular information. Combining these approaches, we achieve Raman imaging speed-ups of up to 40-90×, enabling good-quality cellular imaging with a high-resolution, high signal-to-noise ratio in under 1 min. We further demonstrate Raman imaging speed-up of 160×, useful for lower resolution imaging applications such as the rapid screening of large areas or for spectral pathology. Finally, transfer learning is applied to extend DeepeR from cell to tissue-scale imaging. DeepeR provides a foundation that will enable a host of higher-throughput Raman spectroscopy and molecular imaging applications across biomedicine.

Classification of Soft Tissue Sarcoma Specimens with Raman Spectroscopy as Smart Sensing Technology

Intraoperative confirmation of negative resection margins is an essential component of soft tissue sarcoma surgery. Frozen section examination of samples from the resection bed after excision of sarcomas is the gold standard for intraoperative assessment of margin status. However, it takes time to complete histologic examination of these samples, and the technique does not provide real-time diagnosis in the operating room (OR), which delays completion of the operation. This paper presents a study and development of sensing technology using Raman spectroscopy that could be used for detection and classification of the tumor after resection with negative sarcoma margins in real time. We acquired Raman spectra from samples of sarcoma and surrounding benign muscle, fat, and dermis during surgery and developed (i) a quantitative method (QM) and (ii) a machine learning method (MLM) to assess the spectral patterns and determine if they could accurately identify these tissue types when compared to findings in adjacent H&E-stained frozen sections. High classification accuracy (>85%) was achieved with both methods, indicating that these four types of tissue can be identified using the analytical methodology. A hand-held Raman probe could be employed to further develop the methodology to obtain spectra in the OR to provide real-time in vivo capability for the assessment of sarcoma resection margin status.

A Brief History of Machine Learning in Neurosurgery

The history of machine learning in neurosurgery spans three decades and continues to develop at a rapid pace. The earliest applications of machine learning within neurosurgery were first published in the 1990s as researchers began developing artificial neural networks to analyze structured datasets and supervised tasks. By the turn of the millennium, machine learning had evolved beyond proof-of-concept; algorithms had success detecting tumors in unstructured clinical imaging, and unsupervised learning showed promise for tumor segmentation. Throughout the 2000s, the role of machine learning in neurosurgery was further refined. Well-trained models began to consistently best expert clinicians at brain tumor diagnosis. Additionally, the digitization of the healthcare industry provided ample data for analysis, both structured and unstructured. By the 2010s, the use of machine learning within neurosurgery had exploded. The rapid deployment of an exciting new toolset also led to the growing realization that it may offer marginal benefit at best over conventional logistical regression models for analyzing tabular datasets. Additionally, the widespread adoption of machine learning in neurosurgical clinical practice continues to lag until additional validation can ensure generalizability. Many exciting contemporary applications nonetheless continue to demonstrate the unprecedented potential of machine learning to revolutionize neurosurgery when applied to appropriate clinical challenges.

Machine Learning in Pituitary Surgery

Machine learning applications in neurosurgery are increasingly reported for diverse tasks such as faster and more accurate preoperative diagnosis, enhanced lesion characterization, as well as surgical outcome, complications and healthcare cost prediction. Even though the pertinent literature in pituitary surgery is less extensive with respect to other neurosurgical diseases, past research attempted to answer clinically relevant questions to better assist surgeons and clinicians. In the present chapter we review reported ML applications in pituitary surgery including differential diagnosis, preoperative lesion characterization (immunohistochemistry, cavernous sinus invasion, tumor consistency), surgical outcome and complication predictions (gross total resection, tumor recurrence, and endocrinological remission, cerebrospinal fluid leak, postoperative hyponatremia). Moreover, we briefly discuss from a practical standpoint the current barriers to clinical translation of machine learning research. On the topic of pituitary surgery, published reports can be considered mostly preliminary, requiring larger training populations and strong external validation. Thoughtful selection of clinically relevant outcomes of interest and transversal application of model development pipeline-together with accurate methodological planning and multicenter collaborations-have the potential to overcome current limitations and ultimately provide additional tools for more informed patient management.

Machine Learning Algorithms in Neuroimaging: An Overview

Machine learning (ML) and artificial intelligence (AI) applications in the field of neuroimaging have been on the rise in recent years, and their clinical adoption is increasing worldwide. Deep learning (DL) is a field of ML that can be defined as a set of algorithms enabling a computer to be fed with raw data and progressively discover-through multiple layers of representation-more complex and abstract patterns in large data sets. The combination of ML and radiomics, namely the extraction of features from medical images, has proven valuable, too: Radiomic information can be used for enhanced image characterization and prognosis or outcome prediction. This chapter summarizes the basic concepts underlying ML application for neuroimaging and discusses technical aspects of the most promising algorithms, with a specific focus on Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs), in order to provide the readership with the fundamental theoretical tools to better understand ML in neuroimaging. Applications are highlighted from a practical standpoint in the last section of the chapter, including: image reconstruction and restoration, image synthesis and super-resolution, registration, segmentation, classification, and outcome prediction.

Intraoperative pathologically-calibrated diagnosis of lymph nodes involved by breast cancer cells based on electrical impedance spectroscopy; a prospective diagnostic human model study

BACKGROUND

Nodal status evaluation is a crucial step in determining prognostic factors and managing treatment strategies for breast cancer patients. Preoperative (CNB), intraoperative (SLNB), and even postoperative techniques (Formalin-Fixed Paraffin-Embedded sectioning, FFPE) have definite limitations of precision or sometimes are time-consuming for the result declaration. The primary purpose of this prospective study is to provide a precise complementary system for distinguishing lymph nodes (LNs) involved by cancerous cells in breast cancer patients intraoperatively.

METHODS

The proposed system, Electrical Lymph Scoring(ELS), is designed based on the dielectric properties of the under-test LNs. The system has a needle-shaped 2-electrode probe entered into SLNs or ALNs dissected from patients through standard surgical guidelines. Impedance magnitude in f = 1 kH (Z_1kHz) and Impedance Phase Slope in frequency ranges of 100 kHz-500 kHz (IPS) were then extracted from the impedance spectroscopy data in a cohort study of 77 breast cancer patients(totally 282 dissected LNs) who had been undergone surgery before (n = 55) or after (n = 22) chemical therapies (non-neoadjuvant or neoadjuvant chemotherapy). A new admittance parameter(Y_n^') also proposed for LN detection in neoadjuvant chemotherapy patients.

RESULTS

Considering the permanent pathology result as the gold standard checked by two independent expert pathologists, a significant correlation was observed between the presence of cancerous cells in LNs and individual ranges of the ELS electrical responses. Compared with normal LNs containing fatty ambient and immune cells, LNs involved by cancerous clusters would reduce the Z_1kHz and increase the IPS. These changes correlate with fat metabolism by cancer cells due to their Fatty Acid Oxidation (FAO) in LN, which results in different dielectric properties between high and low-fat content of normal and cancerous LNs, respectively.

CONCLUSIONS

By finding the best correlation between our defined impedimetric parameters and pathological states of tested LNs, a real-time intraoperative detection approach was developed for highly-sensitive (92%, P<0.001) diagnosis of involved sentinel or axillary LNs. The impact of real-time intraoperative scoring of SLNs would make a pre-estimation about the necessity of excising further LNs to help the surgeon for less invasive surgery, especially in the absence of frozen-section equipment.

Shot-noise limited tunable dual-vibrational frequency stimulated Raman scattering microscopy

We present a shot-noise limited SRS implementation providing a >200 mW per excitation wavelength that is optimized for addressing two molecular vibrations simultaneously. As the key to producing a 3 ps laser of different colors out of a single fs-laser (15 nm FWHM), we use ultra-steep angle-tunable optical filters to extract 2 narrow-band Stokes laser beams (1-2 nm & 1-2 ps), which are separated by 100 cm^-1. The center part of the fs-laser is frequency doubled to pump an optical parametric oscillator (OPO). The temporal width of the OPO's output (1 ps) is matched to the Stokes beams and can be tuned from 650-980 nm to address simultaneously two Raman shifts separated by 100 cm^-1 that are located between 500 cm^-1 and 5000 cm^-1. We demonstrate background-free SRS imaging of C-D labeled biological samples (bacteria and Drosophila). Furthermore, high quality virtual stimulated Raman histology imaging of a brain adenocarcinoma is shown for pixel dwell times of 16 µs.

Integration of Raman spectra with transcriptome data in glioblastoma multiforme defines tumour subtypes and predicts patient outcome

Raman spectroscopy is an imaging technique that has been applied to assess molecular compositions of living cells to characterize cell types and states. However, owing to the diverse molecular species in cells and challenges of assigning peaks to specific molecules, it has not been clear how to interpret cellular Raman spectra. Here, we provide firm evidence that cellular Raman spectra (RS) and transcriptomic profiles of glioblastoma can be computationally connected and thus interpreted. We find that the dimensions of high-dimensional RS and transcriptomes can be reduced and connected linearly through a shared low-dimensional subspace. Accordingly, we were able to predict global gene expression profiles by applying the calculated transformation matrix to Raman spectra and vice versa. From these analyses, we extract a minimal gene expression signature associated with specific RS profiles and predictive of disease outcome.

Chatbot for Health Care and Oncology Applications Using Artificial Intelligence and Machine Learning: Systematic Review

Background

Chatbot is a timely topic applied in various fields, including medicine and health care, for human-like knowledge transfer and communication. Machine learning, a subset of artificial intelligence, has been proven particularly applicable in health care, with the ability for complex dialog management and conversational flexibility.

Objective

This review article aims to report on the recent advances and current trends in chatbot technology in medicine. A brief historical overview, along with the developmental progress and design characteristics, is first introduced. The focus will be on cancer therapy, with in-depth discussions and examples of diagnosis, treatment, monitoring, patient support, workflow efficiency, and health promotion. In addition, this paper will explore the limitations and areas of concern, highlighting ethical, moral, security, technical, and regulatory standards and evaluation issues to explain the hesitancy in implementation.

Methods

A search of the literature published in the past 20 years was conducted using the IEEE Xplore, PubMed, Web of Science, Scopus, and OVID databases. The screening of chatbots was guided by the open-access Botlist directory for health care components and further divided according to the following criteria: diagnosis, treatment, monitoring, support, workflow, and health promotion.

Results

Even after addressing these issues and establishing the safety or efficacy of chatbots, human elements in health care will not be replaceable. Therefore, chatbots have the potential to be integrated into clinical practice by working alongside health practitioners to reduce costs, refine workflow efficiencies, and improve patient outcomes. Other applications in pandemic support, global health, and education are yet to be fully explored.

Conclusions

Further research and interdisciplinary collaboration could advance this technology to dramatically improve the quality of care for patients, rebalance the workload for clinicians, and revolutionize the practice of medicine.

Label-Free Histology and Evaluation of Human Pancreatic Cancer with Coherent Nonlinear Optical Microscopy

Surgeries achieving maximal tumor resection remain the major effective treatment of pancreatic cancer. Rapid and precise intraoperative diagnosis of pancreatic tissues is critical for optimum surgical outcomes but is challenging for the current staining-based histological methods. We demonstrated that label-free coherent nonlinear optical microscopy with combined stimulated Raman scattering (SRS) and second harmonic generation (SHG) could reveal key diagnostic features of both normal and cancerous human pancreatic tissues. Adjacent pairs of tissue sections from resection margins of 37 patients were imaged by SRS and hematoxylin and eosin staining for direct comparison, demonstrating high diagnostic concordance (Cohen's kappa, κ > 0.97) between them. Fresh unprocessed tissues showed well-preserved histoarchitectures including pancreatic ducts, islets, acini, and nerves. Moreover, the area ratios of collagen fibers were analyzed and found to correlate with the drainage pancreatic amylase level (odds ratio = 28.0, p = 0.0017). Our results indicated that SRS/SHG histology provides potential for rapid intraoperative diagnosis of pancreatic cancer as well as a predictive value of postoperative pancreatic fistula.

Encoding histopathology whole slide images with location-aware graphs for diagnostically relevant regions retrieval

Content-based histopathological image retrieval (CBHIR) has become popular in recent years in histopathological image analysis. CBHIR systems provide auxiliary diagnosis information for pathologists by searching for and returning regions that are contently similar to the region of interest (ROI) from a pre-established database. It is challenging and yet significant in clinical applications to retrieve diagnostically relevant regions from a database consisting of histopathological whole slide images (WSIs). In this paper, we propose a novel framework for regions retrieval from WSI database based on location-aware graphs and deep hash techniques. Compared to the present CBHIR framework, both structural information and global location information of ROIs in the WSI are preserved by graph convolution and self-attention operations, which makes the retrieval framework more sensitive to regions that are similar in tissue distribution. Moreover, benefited from the graph structure, the proposed framework has good scalability for both the size and shape variation of ROIs. It allows the pathologist to define query regions using free curves according to the appearance of tissue. Thirdly, the retrieval is achieved based on the hash technique, which ensures the framework is efficient and adequate for practical large-scale WSI database. The proposed method was evaluated on an in-house endometrium dataset with 2650 WSIs and the public ACDC-LungHP dataset. The experimental results have demonstrated that the proposed method achieved a mean average precision above 0.667 on the endometrium dataset and above 0.869 on the ACDC-LungHP dataset in the task of irregular region retrieval, which are superior to the state-of-the-art methods. The average retrieval time from a database containing 1855 WSIs is 0.752 ms. The source code is available at https://github.com/zhengyushan/lagenet.

Deep Learning on Oral Squamous Cell Carcinoma Ex Vivo Fluorescent Confocal Microscopy Data: A Feasibility Study

BACKGROUND

Ex vivo fluorescent confocal microscopy (FCM) is a novel and effective method for a fast-automatized histological tissue examination. In contrast, conventional diagnostic methods are primarily based on the skills of the histopathologist. In this study, we investigated the potential of convolutional neural networks (CNNs) for automatized classification of oral squamous cell carcinoma via ex vivo FCM imaging for the first time.

MATERIAL AND METHODS

Tissue samples from 20 patients were collected, scanned with an ex vivo confocal microscope immediately after resection, and investigated histopathologically. A CNN architecture (MobileNet) was trained and tested for accuracy.

RESULTS

The model achieved a sensitivity of 0.47 and specificity of 0.96 in the automated classification of cancerous tissue in our study.

CONCLUSION

In this preliminary work, we trained a CNN model on a limited number of ex vivo FCM images and obtained promising results in the automated classification of cancerous tissue. Further studies using large sample sizes are warranted to introduce this technology into clinics.

Stain-free identification of tissue pathology using a generative adversarial network to infer nanomechanical signatures

Intraoperative frozen section analysis can be used to improve the accuracy of tumour margin estimation during cancer resection surgery through rapid processing and pathological assessment of excised tissue. Its applicability is limited in some cases due to the additional risks associated with prolonged surgery, largely from the time-consuming staining procedure. Our work uses a measurable property of bulk tissue to bypass the staining process: as tumour cells proliferate, they influence the surrounding extra-cellular matrix, and the resulting change in elastic modulus provides a signature of the underlying pathology. In this work we accurately localise atomic force microscopy measurements of human liver tissue samples and train a generative adversarial network to infer elastic modulus from low-resolution images of unstained tissue sections. Pathology is predicted through unsupervised clustering of parameters characterizing the distributions of inferred values, achieving 89% accuracy for all samples based on the nominal assessment (n = 28), and 95% for samples that have been validated by two independent pathologists through post hoc staining (n = 20). Our results demonstrate that this technique could increase the feasibility of intraoperative frozen section analysis for use during resection surgery and improve patient outcomes.

Deep-learning based classification distinguishes sarcomatoid malignant mesotheliomas from benign spindle cell mesothelial proliferations

Sarcomatoid mesothelioma is an aggressive malignancy that can be challenging to distinguish from benign spindle cell mesothelial proliferations based on biopsy, and this distinction is crucial to patient treatment and prognosis. A novel deep learning based classifier may be able to aid pathologists in making this critical diagnostic distinction. SpindleMesoNET was trained on cases of malignant sarcomatoid mesothelioma and benign spindle cell mesothelial proliferations. Performance was assessed through cross-validation on the training set, on an independent set of challenging cases referred for expert opinion ('referral' test set), and on an externally stained set from outside institutions ('externally stained' test set). SpindleMesoNET predicted the benign or malignant status of cases with AUC's of 0.932, 0.925, and 0.989 on the cross-validation, referral and external test sets, respectively. The accuracy of SpindleMesoNET on the referral set cases (92.5%) was comparable to the average accuracy of 3 experienced pathologists on the same slide set (91.7%). We conclude that SpindleMesoNET can accurately distinguish sarcomatoid mesothelioma from benign spindle cell mesothelial proliferations. A deep learning system of this type holds potential for future use as an ancillary test in diagnostic pathology.

PBTNet: A New Computer-Aided Diagnosis System for Detecting Primary Brain Tumors

Brain tumors are among the leading human killers. There are over 120 different types of brain tumors, but they mainly fall into two groups: primary brain tumors and metastatic brain tumors. Primary brain tumors develop from normal brain cells. Early and accurate detection of primary brain tumors is vital for the treatment of this disease. Magnetic resonance imaging is the most common method to diagnose brain diseases, but the manual interpretation of the images suffers from high inter-observer variance. In this paper, we presented a new computer-aided diagnosis system named PBTNet for detecting primary brain tumors in magnetic resonance images. A pre-trained ResNet-18 was selected as the backbone model in our PBTNet, but it was fine-tuned only for feature extraction. Then, three randomized neural networks, Schmidt neural network, random vector functional-link, and extreme learning machine served as the classifiers in the PBTNet, which were trained with the features and their labels. The final predictions of the PBTNet were generated by the ensemble of the outputs from the three classifiers. 5-fold cross-validation was employed to evaluate the classification performance of the PBTNet, and experimental results demonstrated that the proposed PBTNet was an effective tool for the diagnosis of primary brain tumors.

A Novel and Rapid Serum Detection Technology for Non-Invasive Screening of Gastric Cancer Based on Raman Spectroscopy Combined With Different Machine Learning Methods

Gastric cancer (GC) is the fifth most common cancer in the world and a serious threat to human health. Due to its high morbidity and mortality, a simple, rapid and accurate early screening method for GC is urgently needed. In this study, the potential of Raman spectroscopy combined with different machine learning methods was explored to distinguish serum samples from GC patients and healthy controls. Serum Raman spectra were collected from 109 patients with GC (including 35 in stage I, 14 in stage II, 35 in stage III, and 25 in stage IV) and 104 healthy volunteers matched for age, presenting for a routine physical examination. We analyzed the difference in serum metabolism between GC patients and healthy people through a comparative study of the average Raman spectra of the two groups. Four machine learning methods, one-dimensional convolutional neural network, random forest, support vector machine, and K-nearest neighbor were used to explore identifying two sets of Raman spectral data. The classification model was established by using 70% of the data as a training set and 30% as a test set. Using unseen data to test the model, the RF model yielded an accuracy of 92.8%, and the sensitivity and specificity were 94.7% and 90.8%. The performance of the RF model was further confirmed by the receiver operating characteristic (ROC) curve, with an area under the curve (AUC) of 0.9199. This exploratory work shows that serum Raman spectroscopy combined with RF has great potential in the machine-assisted classification of GC, and is expected to provide a non-destructive and convenient technology for the screening of GC patients.

Annotation-efficient deep learning for automatic medical image segmentation

Automatic medical image segmentation plays a critical role in scientific research and medical care. Existing high-performance deep learning methods typically rely on large training datasets with high-quality manual annotations, which are difficult to obtain in many clinical applications. Here, we introduce Annotation-effIcient Deep lEarning (AIDE), an open-source framework to handle imperfect training datasets. Methodological analyses and empirical evaluations are conducted, and we demonstrate that AIDE surpasses conventional fully-supervised models by presenting better performance on open datasets possessing scarce or noisy annotations. We further test AIDE in a real-life case study for breast tumor segmentation. Three datasets containing 11,852 breast images from three medical centers are employed, and AIDE, utilizing 10% training annotations, consistently produces segmentation maps comparable to those generated by fully-supervised counterparts or provided by independent radiologists. The 10-fold enhanced efficiency in utilizing expert labels has the potential to promote a wide range of biomedical applications.

Artificial Intelligence in Brain Tumour Surgery-An Emerging Paradigm

Artificial intelligence (AI) platforms have the potential to cause a paradigm shift in brain tumour surgery. Brain tumour surgery augmented with AI can result in safer and more effective treatment. In this review article, we explore the current and future role of AI in patients undergoing brain tumour surgery, including aiding diagnosis, optimising the surgical plan, providing support during the operation, and better predicting the prognosis. Finally, we discuss barriers to the successful clinical implementation, the ethical concerns, and we provide our perspective on how the field could be advanced.

Fluorescence Lifetime Imaging and Spectroscopic Co-Validation for Protoporphyrin IX-Guided Tumor Visualization in Neurosurgery

Maximal safe resection is a key strategy for improving patient prognosis in the management of brain tumors. Intraoperative fluorescence guidance has emerged as a standard in the surgery of high-grade gliomas. The administration of 5-aminolevulinic acid prior to surgery induces tumor-specific accumulation of protoporphyrin IX, which emits red fluorescence under blue-light illumination. The technology, however, is substantially limited for low-grade gliomas and weakly tumor-infiltrated brain, where low protoporphyrin IX concentrations are outweighed by tissue autofluorescence. In this context, fluorescence lifetime imaging has shown promise to distinguish spectrally overlapping fluorophores. We integrated frequency-domain fluorescence lifetime imaging in a surgical microscope and combined it with spatially registered fluorescence spectroscopy, which can be considered a research benchmark for sensitive protoporphyrin IX detection. Fluorescence lifetime maps and spectra were acquired for a representative set of fresh ex-vivo brain tumor specimens (low-grade gliomas n = 15, high-grade gliomas n = 80, meningiomas n = 41, and metastases n = 35). Combining the fluorescence lifetime with fluorescence spectra unveiled how weak protoporphyrin IX accumulations increased the lifetime respective to tissue autofluorescence. Infiltration zones (4.1ns ± 1.8ns, p = 0.017) and core tumor areas (4.8ns ± 1.3ns, p = 0.040) of low-grade gliomas were significantly distinguishable from non-pathologic tissue (1.6ns ± 0.5ns). Similarly, fluorescence lifetimes for infiltrated and reactive tissue as well as necrotic and core tumor areas were increased for high-grade gliomas and metastasis. Meningioma tumor specimens showed strongly increased lifetimes (12.2ns ± 2.5ns, p = 0.005). Our results emphasize the potential of fluorescence lifetime imaging to optimize maximal safe resection in brain tumors in future and highlight its potential toward clinical translation.

Introduction to The Spine Journal special issue on artificial intelligence and machine learning

In the last 5 years, artificial intelligence (AI) algorithms have made rapid advances for diagnosis and prognosis in fields ranging from dermatology to anesthesiology. How do we make sense of the rise of AI in healthcare and specifically in spine? How much of what we see today is "hype" and what will remain when the dust settles? In this special issue, several reviews and original articles help us understand the state of AI in healthcare today, the avenues for future progress, and the implications for spine care.  Continued engagement, skepticism, and collaboration with technical experts will allow for the development of AI systems that complement and expand our abilities to diagnose, predict, and operate.

Multi-color stimulated Raman scattering with a frame-to-frame wavelength-tunable fiber-based light source

We present multi-color imaging by stimulated Raman scattering (SRS) enabled by an ultrafast fiber-based light source with integrated amplitude modulation and frame-to-frame wavelength tuning. With a relative intensity noise level of -153.7 dBc/Hz at 20.25 MHz the light source is well suited for SRS imaging and outperforms other fiber-based light source concepts for SRS imaging. The light source is tunable in under 5 ms per arbitrary wavelength step between 700 cm^-1 and 3200 cm^-1, which allows for addressing Raman resonances from the fingerprint to the CH-stretch region. Moreover, the compact and environmentally stable system is predestined for fast multi-color assessments of medical or rapidly evolving samples with high chemical specificity, paving the way for diagnostics and sensing outside of specialized laser laboratories.

SMART on FHIR in spine: integrating clinical prediction models into electronic health records for precision medicine at the point of care

Recent applications of artificial intelligence have shown great promise for improving the quality and efficiency of clinical care. Numerous clinical decision support tools exist in today's electronic health records (EHRs) such as medication dosing support, order facilitators (eg, procedure specific order sets), and point of care alerts. However, less has been done to integrate artificial intelligence (AI)-enabled risk predictors into EHRs despite wide availability of validated risk prediction tools. An interoperability standard known as SMART on FHIR (substitutable medical applications and reusable technologies on fast health interoperability resources) offers a promising path forward, enabling digital innovations to be seamlessly integrated with the EHR with regard to the user interface and patient data. For the next step in progress towards the goal of learning healthcare and informatics-enabled spine surgery, we propose the application of SMART on FHIR to integrate existing and new risk predictions tools in spine surgery through an EHR add-on-application.

Metabolic profiles of human brain parenchyma and glioma for rapid tissue diagnosis by targeted desorption electrospray ionization mass spectrometry

Desorption electrospray ionization mass spectrometry (DESI-MS) is well suited for intraoperative tissue analysis since it requires little sample preparation and offers rapid and sensitive molecular diagnostics. Currently, intraoperative assessment of the tumor cell percentage of glioma biopsies can be made by measuring a single metabolite, N-acetylaspartate (NAA). The inclusion of additional biomarkers will likely improve the accuracy when distinguishing brain parenchyma from glioma by DESI-MS. To explore this possibility, mass spectra were recorded for extracts from 32 unmodified human brain samples with known pathology. Statistical analysis of data obtained from full-scan and multiple reaction monitoring (MRM) profiles identified discriminatory metabolites, namely gamma-aminobutyric acid (GABA), creatine, glutamic acid, carnitine, and hexane-1,2,3,4,5,6-hexol (abbreviated as hexol), as well as the established biomarker NAA. Brain parenchyma was readily differentiated from glioma based on these metabolites as measured both in full-scan mass spectra and by the intensities of their characteristic MRM transitions. New DESI-MS methods (5 min acquisition using full scans and MS/MS), developed to measure ion abundance ratios among these metabolites, were tested using smears of 29 brain samples. Ion abundance ratios based on signals for GABA, creatine, carnitine, and hexol all had sensitivities > 90%, specificities > 80%, and accuracies > 85%. Prospectively, the implementation of diagnostic ion abundance ratios should strengthen the discriminatory power of individual biomarkers and enhance method robustness against signal fluctuations, resulting in an improved DESI-MS method of glioma diagnosis.

Detection of pancreatic cancer by convolutional-neural-network-assisted spontaneous Raman spectroscopy with critical feature visualization

Pancreatic cancer is the deadliest cancer type with a five-year survival rate of less than 9%. Detection of tumor margins plays an essential role in the success of surgical resection. However, histopathological assessment is time-consuming, expensive, and labor-intensive. We constructed a lab-designed, hand-held Raman spectroscopic system that could enable intraoperative tissue diagnosis using convolutional neural network (CNN) models to efficiently distinguish between cancerous and normal pancreatic tissue. To our best knowledge, this is the first reported effort to diagnose pancreatic cancer by CNN-aided spontaneous Raman scattering with a lab-developed system designed for intraoperative applications. Classification based on the original one-dimensional (1D) Raman, two-dimensional (2D) Raman images, and the first principal component (PC1) from the principal component analysis on the 2D image, could all achieve high performance: the testing sensitivity, specificity, and accuracy were over 95%, and the area under the curve approached 0.99. Although CNN models often show great success in classification, it has always been challenging to visualize the CNN features in these models, which has never been achieved in the Raman spectroscopy application in cancer diagnosis. By studying individual Raman regions and by extracting and visualizing CNN features from max-pooling layers, we identified critical Raman peaks that could aid in the classification of cancerous and noncancerous tissues. 2D Raman PC1 yielded more critical peaks for pancreatic cancer identification than that of 1D Raman, as the Raman intensity was amplified by 2D Raman PC1. To our best knowledge, the feature visualization was achieved for the first time in the field of CNN-aided spontaneous Raman spectroscopy for cancer diagnosis. Based on these CNN feature peaks and their frequency at specific wavenumbers, pancreatic cancerous tissue was found to contain more biochemical components related to the protein contents (particularly collagen), whereas normal pancreatic tissue was found to contain more lipids and nucleic acid (particularly deoxyribonucleic acid/ribonucleic acid). Overall, the CNN model in combination with Raman spectroscopy could serve as a useful tool for the extraction of key features that can help differentiate pancreatic cancer from a normal pancreas.

Neurosurgical Advances for Malignant Gliomas: Intersection of Biology and Technology

ABSTRACT

The intersection of biology and technology has led to many advancements for the field of neurosurgery. Molecular developments have led to the identification of specific mutations, allowing for more accurate discussions in regard to prognosis and treatment effect. Even amid the progress from basic science benchwork, malignant gliomas continue to have a bleak natural history in lieu of the resistance to chemotherapy and the diffuse nature of the disease, leaving room for further research to discover more effective treatment modalities. Novel imaging methods, including the emerging field of radiogenomics, involve the merging of molecular and radiographic data, enabling earlier, detailed molecular diagnoses and improved surveillance of this pathology. Furthermore, surgical advancements have led to safer and more extensive resections. This review aims to delineate the various advancements in the many facets that are used daily in the care of our glioma population, specifically pertaining to its biology, imaging modalities, and perioperative adjuncts used in the operating room.

Accurate diagnosis of lung tissues for 2D Raman spectrogram by deep learning based on short-time Fourier transform

Multivariate statistical analysis methods have an important role in spectrochemical analyses to rapidly identify and diagnose cancer and the subtype. However, utilizing these methods to analyze lager amount spectral data is challenging, and poses a major bottleneck toward achieving high accuracy. Here, a new convolutional neural networks (CNN) method based on short-time Fourier transform (STFT) to diagnose lung tissues via Raman spectra readily is proposed. The models yield that the accuracies of the new method are higher than the conventional methods (principal components analysis -linear discriminant analysis and support vector machine) for validation group (95.2% vs 85.5%, 94.4%) and test group (96.5% vs 90.4%, 93.9%) after cross-validation. The results illustrate that the new method which converts one-dimensional Raman data into two-dimensional Raman spectrograms improve the discriminatory ability of lung tissues and can achieve automatically accurate diagnosis of lung tissues.

Artificial intelligence-enhanced intraoperative neurosurgical workflow: state of the art and future perspectives

BACKGROUND

Artificial Intelligence (AI) and Machine Learning (ML) augment decision-making processes and productivity by supporting surgeons over a range of clinical activities: from diagnosis and preoperative planning to intraoperative surgical assistance. We reviewed the literature to identify current AI platforms applied to neurosurgical perioperative and intraoperative settings and describe their role in multiple subspecialties.

METHODS

A systematic review of the literature was conducted following the PRISMA guidelines. PubMed, EMBASE, and Scopus databases were searched from inception to December 31, 2020. Original articles were included if they: presented AI platforms implemented in perioperative, intraoperative settings and reported ML models' performance metrics. Due to the heterogeneity in neurosurgical applications, a qualitative synthesis was deemed appropriate. The risk of bias and applicability of predicted outcomes were assessed using the PROBAST tool.

RESULTS

41 articles were included. All studies evaluated a supervised learning algorithm. A total of 10 ML models were described; the most frequent were neural networks (n = 15) and tree-based models (n = 13). Overall, the risk of bias was medium-high, but applicability was considered positive for all studies. Articles were grouped into 4 categories according to the subspecialty of interest: neuro-oncology, spine, functional and other. For each category, different prediction tasks were identified.

CONCLUSIONS

In this review, we summarize the state-of-art applications of AI for the intraoperative augmentation of neurosurgical workflows across multiple subspecialties. ML models may boost surgical team performances by reducing human errors and providing patient-tailored surgical plans, but further and higher-quality studies need to be conducted.

Quantitative analysis of lithium in brine by laser-induced breakdown spectroscopy based on convolutional neural network

In this study, a simple and effective method for accurate determination of lithium in brine samples was developed by the combination of laser induced breakdown spectroscopy (LIBS) and convolutional neural network (CNN). Our results clearly demonstrate that the use of CNN could efficiently overcome the complex matrix effects, and thus allows for on-site Li quantitative determination in brine samples by LIBS. Specifically, two CNN models with different input data (M-CNN with matrix emission lines, and DP-CNN with double Li lines) were constructed based on the primary matrix features on spectrum and Boltzmann equation, respectively. It was observed that DP-CNN model could greatly improve the accuracy of Li analysis. We also compared the quantitative analysis capabilities of DP-CNN model with partial least squares regression (PLSR) and principal component analysis-support vector regression (PCA-SVR) model, and the results clearly showed DP-CNN offers the best quantification results (higher accuracy and less matrix interference). Finally, five real brine samples were successfully analyzed by the proposed DP-CNN model, confirming by the average absolute error of the prediction of 0.28 mg L^-1 and the average relative error of 3.48%. These results clearly demonstrate that input data plays an important role in the training of CNN model in LIBS analysis, and the proposed DP-CNN provides an effective approach to solve the matrix effects encountered in LIBS for Li measurement in brine samples.

Recent Advances in Electrochemical Biosensors: Applications, Challenges, and Future Scope

The electrochemical biosensors are a class of biosensors which convert biological information such as analyte concentration that is a biological recognition element (biochemical receptor) into current or voltage. Electrochemical biosensors depict propitious diagnostic technology which can detect biomarkers in body fluids such as sweat, blood, feces, or urine. Combinations of suitable immobilization techniques with effective transducers give rise to an efficient biosensor. They have been employed in the food industry, medical sciences, defense, studying plant biology, etc. While sensing complex structures and entities, a large data is obtained, and it becomes difficult to manually interpret all the data. Machine learning helps in interpreting large sensing data. In the case of biosensors, the presence of impurity affects the performance of the sensor and machine learning helps in removing signals obtained from the contaminants to obtain a high sensitivity. In this review, we discuss different types of biosensors along with their applications and the benefits of machine learning. This is followed by a discussion on the challenges, missing gaps in the knowledge, and solutions in the field of electrochemical biosensors. This review aims to serve as a valuable resource for scientists and engineers entering the interdisciplinary field of electrochemical biosensors. Furthermore, this review provides insight into the type of electrochemical biosensors, their applications, the importance of machine learning (ML) in biosensing, and challenges and future outlook.

Coherent Raman scattering imaging with a near-infrared achromatic metalens

Miniature handheld imaging devices and endoscopes based on coherent Raman scattering are promising for label-free in vivo optical diagnosis. Toward the development of these small-scale systems, a challenge arises from the design and fabrication of achromatic and high-end miniature optical components for both pump and Stokes laser wavelengths. Here, we report a metasurface converting a low-cost plano-convex lens into a water-immersion, nearly diffraction-limited and achromatic lens. The metasurface comprising amorphous silicon nanopillars is designed in a way that all incident rays arrive at the focus with the same phase and group delay, leading to corrections of monochromatic and chromatic aberrations of the refractive lens, respectively. Compared to the case without the metasurface, the hybrid metasurface-refractive lens has higher Strehl ratios than the plano-convex lens and a tighter depth of focus. The hybrid metasurface-refractive lens is utilized in spectroscopic stimulated Raman scattering and coherent anti-Stokes Raman scattering imaging for the differentiation of two different polymer microbeads. Subsequently, the hybrid metalens is harnessed for volumetric coherent Raman scattering imaging of bead and tissue samples. Finally, we discuss possible approaches to integrate such hybrid metalens in a miniature scanning system for label-free coherent Raman scattering endoscopes.

Deep-learning-assisted microscopy with ultraviolet surface excitation for rapid slide-free histological imaging

Histopathological examination of tissue sections is the gold standard for disease diagnosis. However, the conventional histopathology workflow requires lengthy and laborious sample preparation to obtain thin tissue slices, causing about a one-week delay to generate an accurate diagnostic report. Recently, microscopy with ultraviolet surface excitation (MUSE), a rapid and slide-free imaging technique, has been developed to image fresh and thick tissues with specific molecular contrast. Here, we propose to apply an unsupervised generative adversarial network framework to translate colorful MUSE images into Deep-MUSE images that highly resemble hematoxylin and eosin staining, allowing easy adaptation by pathologists. By eliminating the needs of all sample processing steps (except staining), a MUSE image with subcellular resolution for a typical brain biopsy (5 mm × 5 mm) can be acquired in 5 minutes, which is further translated into a Deep-MUSE image in 40 seconds, simplifying the standard histopathology workflow dramatically and providing histological images intraoperatively.

Unsupervised Resolution of Histomorphologic Heterogeneity in Renal Cell Carcinoma Using a Brain Tumor-Educated Neural Network

PURPOSE

Applications of deep learning to histopathology have proven capable of expert-level performance, but approaches have largely focused on supervised classification tasks requiring context-specific training and deployment. More generalizable workflows that can be easily shared across subspecialties could help accelerate and broaden adoption. Here, we hypothesized that histology-optimized feature representations, generated by a convolutional neural network (CNN) during supervised learning, are transferable and can resolve meaningful differences in large-scale, discovery-type unsupervised analyses.

METHODS

We used a CNN, previously trained to recognize brain tumor histomorphologies, to extract 512 feature representations from > 550 digital whole-slide images (WSIs) of renal cell carcinomas (RCCs) from The Cancer Genome Atlas and other previously unencountered tumors. We use these extracted feature vectors to conduct unsupervised image-set clustering and analyze the clinical and biologic relevance of the intra- and interpatient subgroups generated.

RESULTS

Within individual WSIs, feature-based clustering could reliably segment tumor regions and other relevant histopathologic subpatterns (eg, adenosquamous and poorly differentiated regions). Across the larger RCC cohorts, clustering extracted features generated subgroups enriched for clinically relevant subtypes (eg, papillary RCC) and outcomes (eg, survival). Importantly, individual feature activation mapping highlighted salient subtype-specific patterns and features of malignancies (eg, nuclear grade, sarcomatous change) contributing to subgroupings. Moreover, some proposed clusters were enriched for recurring, human-based RCC-subtype misclassifications.

CONCLUSION

Our data support that CNNs, pretrained on large histologic datasets, can extend learned representations to novel scenarios and resolve clinically relevant intra- and interpatient tissue-pattern differences without explicit instruction or additional optimization. Repositioning of existing histology-educated networks could provide scalable approaches for image classification, quality assurance, and discovery of unappreciated patterns and subgroups of disease.

Machine Learning in Oncology: What Should Clinicians Know?

The volume and complexity of scientific and clinical data in oncology have grown markedly over recent years, including but not limited to the realms of electronic health data, radiographic and histologic data, and genomics. This growth holds promise for a deeper understanding of malignancy and, accordingly, more personalized and effective oncologic care. Such goals require, however, the development of new methods to fully make use of the wealth of available data. Improvements in computer processing power and algorithm development have positioned machine learning, a branch of artificial intelligence, to play a prominent role in oncology research and practice. This review provides an overview of the basics of machine learning and highlights current progress and challenges in applying this technology to cancer diagnosis, prognosis, and treatment recommendations, including a discussion of current takeaways for clinicians.

Uncovering Spatiotemporal Heterogeneity of High-Grade Gliomas: From Disease Biology to Therapeutic Implications

Glioblastomas (GBM) are the most common and aggressive tumors of the central nervous system. Rapid tumor growth and diffuse infiltration into healthy brain tissue, along with high intratumoral heterogeneity, challenge therapeutic efficacy and prognosis. A better understanding of spatiotemporal tumor heterogeneity at the histological, cellular, molecular, and dynamic levels would accelerate the development of novel treatments for this devastating brain cancer. Histologically, GBM is characterized by nuclear atypia, cellular pleomorphism, necrosis, microvascular proliferation, and pseudopalisades. At the cellular level, the glioma microenvironment comprises a heterogeneous landscape of cell populations, including tumor cells, non-transformed/reactive glial and neural cells, immune cells, mesenchymal cells, and stem cells, which support tumor growth and invasion through complex network crosstalk. Genomic and transcriptomic analyses of gliomas have revealed significant inter and intratumoral heterogeneity and insights into their molecular pathogenesis. Moreover, recent evidence suggests that diverse dynamics of collective motion patterns exist in glioma tumors, which correlate with histological features. We hypothesize that glioma heterogeneity is not stochastic, but rather arises from organized and dynamic attributes, which favor glioma malignancy and influences treatment regimens. This review highlights the importance of an integrative approach of glioma histopathological features, single-cell and spatially resolved transcriptomic and cellular dynamics to understand tumor heterogeneity and maximize therapeutic effects.