Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks

Artificial intelligence to assist decision-making on pharmacotherapy: A feasibility study

Background

Artificial intelligence (AI) has the capability to analyze vast amounts of data and has been applied in various healthcare sectors. However, its effectiveness in aiding pharmacotherapy decision-making remains uncertain due to the intricate, patient-specific, and dynamic nature of this field.

Objective

This study sought to investigate the potential of AI in guiding pharmacotherapy decisions using clinical data such as diagnoses, laboratory results, and vital signs obtained from routine patient care.

Methods

Data of a previous study on medication therapy optimization was updated and adapted for the purpose of this study. Analysis was conducted using R software along with the tidymodels extension packages. The dataset was split into 74% for training and 26% for testing. Decision trees were selected as the primary model due to their simplicity, transparency, and interpretability. To prevent overfitting, bootstrapping techniques were employed, and hyperparameters were fine-tuned. Performance metrics such as areas under the curve and accuracies were computed.

Results

The study cohort comprised 101 elderly patients with multiple diagnoses and complex medication regimens. The AI model demonstrated prediction accuracies ranging from 38% to 100% for various cardiovascular drug classes. Laboratory data and vital signs could not be interpreted, as the effect and dependence were unclear for the model. The study revealed that the issue of AI lag time in responding to sudden changes could be addressed by manually adjusting decision trees, a task not feasible with neural networks.

Conclusion

In conclusion, the AI model exhibited promise in recommending appropriate medications for individual patients. While the study identified several obstacles during model development, most were successfully resolved. Future AI studies need to include the drug effect, not only the drug, if laboratory data is part of the decision. This could assist with interpreting their potential relationship. Human oversight and intervention remain essential for an AI-driven pharmacotherapy decision support system to ensure safe and effective patient care.

Progress and trends in neurological disorders research based on deep learning

In recent years, deep learning (DL) has emerged as a powerful tool in clinical imaging, offering unprecedented opportunities for the diagnosis and treatment of neurological disorders (NDs). This comprehensive review explores the multifaceted role of DL techniques in leveraging vast datasets to advance our understanding of NDs and improve clinical outcomes. Beginning with a systematic literature review, we delve into the utilization of DL, particularly focusing on multimodal neuroimaging data analysis-a domain that has witnessed rapid progress and garnered significant scientific interest. Our study categorizes and critically analyses numerous DL models, including Convolutional Neural Networks (CNNs), LSTM-CNN, GAN, and VGG, to understand their performance across different types of Neurology Diseases. Through particular analysis, we identify key benchmarks and datasets utilized in training and testing DL models, shedding light on the challenges and opportunities in clinical neuroimaging research. Moreover, we discuss the effectiveness of DL in real-world clinical scenarios, emphasizing its potential to revolutionize ND diagnosis and therapy. By synthesizing existing literature and describing future directions, this review not only provides insights into the current state of DL applications in ND analysis but also covers the way for the development of more efficient and accessible DL techniques. Finally, our findings underscore the transformative impact of DL in reshaping the landscape of clinical neuroimaging, offering hope for enhanced patient care and groundbreaking discoveries in the field of neurology. This review paper is beneficial for neuropathologists and new researchers in this field.

The oncological role of resection in newly diagnosed diffuse adult-type glioma defined by the WHO 2021 classification: a Review by the RANO resect group

Glioma resection is associated with prolonged survival, but neuro-oncological trials have frequently refrained from quantifying the extent of resection. The Response Assessment in Neuro-Oncology (RANO) resect group is an international, multidisciplinary group that aims to standardise research practice by delineating the oncological role of surgery in diffuse adult-type gliomas as defined per WHO 2021 classification. Favourable survival effects of more extensive resection unfold over months to decades depending on the molecular tumour profile. In tumours with a more aggressive natural history, supramaximal resection might correlate with additional survival benefit. Weighing the expected survival benefits of resection as dictated by molecular tumour profiles against clinical factors, including the introduction of neurological deficits, we propose an algorithm to estimate the oncological effects of surgery for newly diagnosed gliomas. The algorithm serves to select patients who might benefit most from extensive resection and to emphasise the relevance of quantifying the extent of resection in clinical trials.

Fast intraoperative detection of primary CNS lymphoma and differentiation from common CNS tumors using stimulated Raman histology and deep learning

Accurate intraoperative diagnosis is crucial for differentiating between primary CNS lymphoma (PCNSL) and other CNS entities, guiding surgical decision-making, but represents significant challenges due to overlapping histomorphological features, time constraints, and differing treatment strategies. We combined stimulated Raman histology (SRH) with deep learning to address this challenge. We imaged unprocessed, label-free tissue samples intraoperatively using a portable Raman scattering microscope, generating virtual H&E-like images within less than three minutes. We developed a deep learning pipeline called RapidLymphoma based on a self-supervised learning strategy to (1) detect PCNSL, (2) differentiate from other CNS entities, and (3) test the diagnostic performance in a prospective international multicenter cohort and two additional independent test cohorts. We trained on 54,000 SRH patch images sourced from surgical resections and stereotactic-guided biopsies, including various CNS tumor/non-tumor lesions. Training and test data were collected from four tertiary international medical centers. The final histopathological diagnosis served as ground-truth. In the prospective test cohort of PCNSL and non-PCNSL entities (n=160), RapidLymphoma achieved an overall balanced accuracy of 97.81% ±0.91, non-inferior to frozen section analysis in detecting PCNSL (100% vs. 78.94%). The additional test cohorts (n=420, n=59) reached balanced accuracy rates of 95.44% ±0.74 and 95.57% ±2.47 in differentiating IDH-wildtype diffuse gliomas and various brain metastasis from PCNSL. Visual heatmaps revealed RapidLymphoma's capabilities to detect class-specific histomorphological key features. RapidLymphoma is valid and reliable in detecting PCNSL and differentiating from other CNS entities within three minutes, as well as visual feedback in an intraoperative setting. This leads to fast clinical decision-making and further treatment strategy planning.

A Dual-Modal, Label-Free Raman Imaging Method for Rapid Virtual Staining of Large-Area Breast Cancer Tissue Sections

As one of the most common cancers, accurate, rapid, and simple histopathological diagnosis is very important for breast cancer. Raman imaging is a powerful technique for label-free analysis of tissue composition and histopathology, but it suffers from slow speed when applied to large-area tissue sections. In this study, we propose a dual-modal Raman imaging method that combines Raman mapping data with microscopy bright-field images to achieve virtual staining of breast cancer tissue sections. We validate our method on various breast tissue sections with different morphologies and biomarker expressions and compare it with the golden standard of histopathological methods. The results demonstrate that our method can effectively distinguish various types and components of tissues, and provide staining images comparable to stained tissue sections. Moreover, our method can improve imaging speed by up to 65 times compared to general spontaneous Raman imaging methods. It is simple, fast, and suitable for clinical applications.

Accurate MRI-Based Brain Tumor Diagnosis: Integrating Segmentation and Deep Learning Approaches

Magnetic Resonance Imaging (MRI) is vital in diagnosing brain tumours, offering crucial insights into tumour morphology and precise localisation. Despite its pivotal role, accurately classifying brain tumours from MRI scans is inherently complex due to their heterogeneous characteristics. This study presents a novel integration of advanced segmentation methods with deep learning ensemble algorithms to enhance the classification accuracy of MRI-based brain tumour diagnosis. We conduct a thorough review of both traditional segmentation approaches and contemporary advancements in region-based and machine learning-driven segmentation techniques. This paper explores the utility of deep learning ensemble algorithms, capitalising on the diversity of model architectures to augment tumour classification accuracy and robustness. Through the synergistic amalgamation of sophisticated segmentation techniques and ensemble learning strategies, this research addresses the shortcomings of traditional methodologies, thereby facilitating more precise and efficient brain tumour classification.

SERS surgical navigation with postsurgical immunotherapy of local microtumors and distant metastases for improved anticancer outcomes

The standard of clinical care of most malignant solid cancers is surgery, followed by postsurgical adjuvant therapy, but microtumor lesions left behind after surgery and invisible distant metastases are the major reasons for treatment failure. Here, we report an integrated strategy combining surface-enhanced Raman spectroscopy (SERS) surgical navigation with postsurgical immunotherapy elicited by near-infrared II photothermal treatment and programmed death-1 antibody. The SERS surgical navigation is principally based on the multifunctional optical probes (namely, MATRA probes) integrating with T1-weighted magnetic resonance (MR) imaging, photothermal effect and Raman spectroscopic detection. We demonstrate in a 4T1 breast tumor mouse model that the pre-surgical MR/SERS dual-modal imaging is capable of providing comprehensive tumor information, and intraoperative SERS detection allows accurately delineating the tumor margins and guiding the surgical resection in real time with the least residual microscopic foci. We verify that the postsurgical immunotherapy effectively eradicates those local microtumor lesions and invisible distant metastases, greatly inhibiting the postsurgical cancer recurrence and distant metastasis.

Intraoperative AI-assisted early prediction of parathyroid and ischemia alert in endoscopic thyroid surgery

BACKGROUND

The preservation of parathyroid glands is crucial in endoscopic thyroid surgery to prevent hypocalcemia and related complications. However, current methods for identifying and protecting these glands have limitations. We propose a novel technique that has the potential to improve the safety and efficacy of endoscopic thyroid surgery.

PURPOSE

Our study aims to develop a deep learning model called PTAIR 2.0 (Parathyroid gland Artificial Intelligence Recognition) to enhance parathyroid gland recognition during endoscopic thyroidectomy. We compare its performance against traditional surgeon-based identification methods.

MATERIALS AND METHODS

Parathyroid tissues were annotated in 32 428 images extracted from 838 endoscopic thyroidectomy videos, forming the internal training cohort. An external validation cohort comprised 54 full-length videos. Six candidate algorithms were evaluated to select the optimal one. We assessed the model's performance in terms of initial recognition time, identification duration, and recognition rate and compared it with the performance of surgeons.

RESULTS

Utilizing the YOLOX algorithm, we developed PTAIR 2.0, which demonstrated superior performance with an AP50 score of 92.1%. The YOLOX algorithm achieved a frame rate of 25.14 Hz, meeting real-time requirements. In the internal training cohort, PTAIR 2.0 achieved AP50 values of 94.1%, 98.9%, and 92.1% for parathyroid gland early prediction, identification, and ischemia alert, respectively. Additionally, in the external validation cohort, PTAIR outperformed both junior and senior surgeons in identifying and tracking parathyroid glands (p < 0.001).

CONCLUSION

The AI-driven PTAIR 2.0 model significantly outperforms both senior and junior surgeons in parathyroid gland identification and ischemia alert during endoscopic thyroid surgery, offering potential for enhanced surgical precision and patient outcomes.

Intraoperative near infrared functional imaging of rectal cancer using artificial intelligence methods - now and near future state of the art

Colorectal cancer remains a major cause of cancer death and morbidity worldwide. Surgery is a major treatment modality for primary and, increasingly, secondary curative therapy. However, with more patients being diagnosed with early stage and premalignant disease manifesting as large polyps, greater accuracy in diagnostic and therapeutic precision is needed right from the time of first endoscopic encounter. Rapid advancements in the field of artificial intelligence (AI), coupled with widespread availability of near infrared imaging (currently based around indocyanine green (ICG)) can enable colonoscopic tissue classification and prognostic stratification for significant polyps, in a similar manner to contemporary dynamic radiological perfusion imaging but with the advantage of being able to do so directly within interventional procedural time frames. It can provide an explainable method for immediate digital biopsies that could guide or even replace traditional forceps biopsies and provide guidance re margins (both areas where current practice is only approximately 80% accurate prior to definitive excision). Here, we discuss the concept and practice of AI enhanced ICG perfusion analysis for rectal cancer surgery while highlighting recent and essential near-future advancements. These include breakthrough developments in computer vision and time series analysis that allow for real-time quantification and classification of fluorescent perfusion signals of rectal cancer tissue intraoperatively that accurately distinguish between normal, benign, and malignant tissues in situ endoscopically, which are now undergoing international prospective validation (the Horizon Europe CLASSICA study). Next stage advancements may include detailed digital characterisation of small rectal malignancy based on intraoperative assessment of specific intratumoral fluorescent signal pattern. This could include T staging and intratumoral molecular process profiling (e.g. regarding angiogenesis, differentiation, inflammatory component, and tumour to stroma ratio) with the potential to accurately predict the microscopic local response to nonsurgical treatment enabling personalised therapy via decision support tools. Such advancements are also applicable to the next generation fluorophores and imaging agents currently emerging from clinical trials. In addition, by providing an understandable, applicable method for detailed tissue characterisation visually, such technology paves the way for acceptance of other AI methodology during surgery including, potentially, deep learning methods based on whole screen/video detailing.

Artificial Intelligence and Deep Learning in Revolutionizing Brain Tumor Diagnosis and Treatment: A Narrative Review

The emergence of artificial intelligence (AI) in the medical field holds promise in improving medical management, particularly in personalized strategies for the diagnosis and treatment of brain tumors. However, integrating AI into clinical practice has proven to be a challenge. Deep learning (DL) is very convenient for extracting relevant information from large amounts of data that has increased in medical history and imaging records, which shortens diagnosis time, that would otherwise overwhelm manual methods. In addition, DL aids in automated tumor segmentation, classification, and diagnosis. DL models such as the Brain Tumor Classification Model and the Inception-Resnet V2, or hybrid techniques that enhance these functions and combine DL networks with support vector machine and k-nearest neighbors, identify tumor phenotypes and brain metastases, allowing real-time decision-making and enhancing preoperative planning. AI algorithms and DL development facilitate radiological diagnostics such as computed tomography, positron emission tomography scans, and magnetic resonance imaging (MRI) by integrating two-dimensional and three-dimensional MRI using DenseNet and 3D convolutional neural network architectures, which enable precise tumor delineation. DL offers benefits in neuro-interventional procedures, and the shift toward computer-assisted interventions acknowledges the need for more accurate and efficient image analysis methods. Further research is needed to realize the potential impact of DL in improving these outcomes.

System transferability of Raman-based oesophageal tissue classification using modern machine learning to support multi-centre clinical diagnostics

BACKGROUND

The clinical potential of Raman spectroscopy is well established but has yet to become established in routine oncology workflows. One barrier slowing clinical adoption is a lack of evidence demonstrating that data taken on one spectrometer transfers across to data taken on another spectrometer to provide consistent diagnoses.

METHODS

We investigated multi-centre transferability using human oesophageal tissue. Raman spectra were taken across three different centres with different spectrometers of the same make and model. By using a common protocol, we aimed to minimise the difference in machine learning performance between centres.

RESULTS

61 oesophageal samples from 51 patients were interrogated by Raman spectroscopy at each centre and classified into one of five pathologies. The overall accuracy and log-loss did not significantly vary when a model trained upon data from any one centre was applied to data taken at the other centres. Computational methods to correct for the data during pre-processing were not needed.

CONCLUSION

We have found that when using the same make and model of spectrometer, together with a common protocol, across different centres it is possible to achieve system transferability without the need for additional computational instrument correction.

AI-Assisted Detection of Biomarkers by Sensors and Biosensors for Early Diagnosis and Monitoring

The steady progress in consumer electronics, together with improvement in microflow techniques, nanotechnology, and data processing, has led to implementation of cost-effective, user-friendly portable devices, which play the role of not only gadgets but also diagnostic tools. Moreover, numerous smart devices monitor patients' health, and some of them are applied in point-of-care (PoC) tests as a reliable source of evaluation of a patient's condition. Current diagnostic practices are still based on laboratory tests, preceded by the collection of biological samples, which are then tested in clinical conditions by trained personnel with specialistic equipment. In practice, collecting passive/active physiological and behavioral data from patients in real time and feeding them to artificial intelligence (AI) models can significantly improve the decision process regarding diagnosis and treatment procedures via the omission of conventional sampling and diagnostic procedures while also excluding the role of pathologists. A combination of conventional and novel methods of digital and traditional biomarker detection with portable, autonomous, and miniaturized devices can revolutionize medical diagnostics in the coming years. This article focuses on a comparison of traditional clinical practices with modern diagnostic techniques based on AI and machine learning (ML). The presented technologies will bypass laboratories and start being commercialized, which should lead to improvement or substitution of current diagnostic tools. Their application in PoC settings or as a consumer technology accessible to every patient appears to be a real possibility. Research in this field is expected to intensify in the coming years. Technological advancements in sensors and biosensors are anticipated to enable the continuous real-time analysis of various omics fields, fostering early disease detection and intervention strategies. The integration of AI with digital health platforms would enable predictive analysis and personalized healthcare, emphasizing the importance of interdisciplinary collaboration in related scientific fields.

The Duality of Raman Scattering

ConspectusFirst predicted more than 100 years ago, Raman scattering is a cornerstone of photonics, spectroscopy, and imaging. The conventional framework of understanding Raman scattering was built on Raman cross section σRaman. Carrying a dimension of area, σRaman characterizes the interaction strength between light and molecules during inelastic scattering. The numerical values of σRaman turn out to be many orders of magnitude smaller in comparison to the linear absorption cross sections σAbsorption of similar molecular systems. Such an enormous gap has been the reason for researchers to believe the extremely feeble Raman scattering ever since its discovery. However, this prevailing picture is conceptually problematic or at least incomplete due to the fact that Raman scattering and linear absorption belong to different orders of light-matter interaction.In this Account, we will summarize an alternate way to think about Raman scattering, which we term stimulated response formulation. To capture the third-order interaction nature of Raman scattering, we introduced stimulated Raman cross section, σSRS, defined as the intrinsic molecular property in response to the external photon fluxes. Foremost, experimental measurement of σSRS turns out to be not weak at all or even larger when fairly compared with electronic counterparts of the same order. The analytical expression for σSRS derived from quantum electrodynamics also supports the measurement and proves that σSRS is intrinsically strong. Hence, σRaman and σSRS can be extremely small and large, respectively, for the same molecule at the same time. Our subsequent theoretical studies show that stimulated response formulation can unify spontaneous emission, stimulated emission, spontaneous Raman, and stimulated Raman via eq 10, in a coherent and symmetric way. In particular, an Einstein-coefficient-like equation, eq 12a, was derived, showing that σRaman can be explicitly expressed as σSRS multiplied by an effective photon flux arising from zero-point fluctuation of the vacuum. The feeble vacuum fluctuation hence explains how σSRS can be intrinsically strong while, at the same time, σRaman ends up being many orders of magnitude smaller when both compared to the electronic counterparts. These two sides of the same coin prompted us to propose "the duality of Raman scattering" (Table 1). Finally, this formulation naturally leads to a quantitative treatment of stimulated Raman scattering (SRS) microscopy, providing an intuitive, molecule-centric explanation as to how SRS microscopy can outperform regular Raman microscopy. Hence, as unveiled by the new formulation, a duality of Raman scattering has emerged, with implications for both fundamental science and practical technology.

Intraoperative Multi-Sensor Tissue Differentiation in (Uro-)Oncology - A Short Review

Precise differentiation of pathological tissue during surgery is crucial in oncology. The current gold standard, histopathological analysis, involves delays due to tissue processing, impacting real-time decision-making. Furthermore, it does not always give information about the extent and heterogeneity of the tumorous tissue. This paper examines our research efforts in developing novel multimodal sensors tailored for uro-oncology. These sensors measure optical, electrical, and mechanical tissue properties, aiming to provide comprehensive tissue differentiation during surgery. Along with this, a review of recent advances in the field of intraoperative tissue differentiation is given. Altered physical properties in tumorous tissues are discussed and the suitability of various sensor modalities for detecting these changes is investigated, especially infrared and Raman spectroscopy, and electrical and mechanical measurements. A digital frameworks for spatial localization of measurements within the organ is is crucial for integrating sensor data to achieve comprehensive tissue characterization. Our focus lies in presenting these innovative sensor technologies and their potential to transform intraoperative tissue assessment. By providing real-time information, these sensors could significantly enhance diagnostic precision in urological oncology, potentially improving patient outcomes.Clinical relevance- Supporting real-time decision making during urological surgeries.

In vivo organoid growth monitoring by stimulated Raman histology

Patient-derived tumor organoids have emerged as a crucial tool for assessing the efficacy of chemotherapy and conducting preclinical drug screenings. However, the conventional histological investigation of these organoids necessitates their devitalization through fixation and slicing, limiting their utility to a single-time analysis. Here, we use stimulated Raman histology (SRH) to demonstrate non-destructive, label-free virtual staining of 3D organoids, while preserving their viability and growth. This novel approach provides contrast similar to conventional staining methods, allowing for the continuous monitoring of organoids over time. Our results demonstrate that SRH transforms organoids from one-time use products into repeatable models, facilitating the efficient selection of effective drug combinations. This advancement holds promise for personalized cancer treatment, allowing for the dynamic assessment and optimization of chemotherapy treatments in patient-specific contexts.

Viewing early life without labels: optical approaches for imaging the early embryo†

Embryo quality is an important determinant of successful implantation and a resultant live birth. Current clinical approaches for evaluating embryo quality rely on subjective morphology assessments or an invasive biopsy for genetic testing. However, both approaches can be inherently inaccurate and crucially, fail to improve the live birth rate following the transfer of in vitro produced embryos. Optical imaging offers a potential non-invasive and accurate avenue for assessing embryo viability. Recent advances in various label-free optical imaging approaches have garnered increased interest in the field of reproductive biology due to their ability to rapidly capture images at high resolution, delivering both morphological and molecular information. This burgeoning field holds immense potential for further development, with profound implications for clinical translation. Here, our review aims to: (1) describe the principles of various imaging systems, distinguishing between approaches that capture morphological and molecular information, (2) highlight the recent application of these technologies in the field of reproductive biology, and (3) assess their respective merits and limitations concerning the capacity to evaluate embryo quality. Additionally, the review summarizes challenges in the translation of optical imaging systems into routine clinical practice, providing recommendations for their future development. Finally, we identify suitable imaging approaches for interrogating the mechanisms underpinning successful embryo development.

Rapid and Precise Diagnosis of Retroperitoneal Liposarcoma with Deep-Learned Label-Free Molecular Microscopy

The retroperitoneal liposarcoma (RLPS) is a rare malignancy whose only curative therapy is surgical resection. However, well-differentiated liposarcomas (WDLPSs), one of its most common types, can hardly be distinguished from normal fat during operation without an effective margin assessment method, jeopardizing the prognosis severely with a high recurrence risk. Here, we combined dual label-free nonlinear optical modalities, stimulated Raman scattering (SRS) microscopy and second harmonic generation (SHG) microscopy, to image two predominant tissue biomolecules, lipids and collagen fibers, in 35 RLPSs and 34 normal fat samples collected from 35 patients. The produced dual-modal tissue images were used for RLPS diagnosis based on deep learning. Dramatically decreasing lipids and increasing collagen fibers during tumor progression were reflected. A ResNeXt101-based model achieved 94.7% overall accuracy and 0.987 mean area under the ROC curve (AUC) in differentiating among normal fat, WDLPSs, and dedifferentiated liposarcomas (DDLPSs). In particular, WDLPSs were detected with 94.1% precision and 84.6% sensitivity superior to existing methods. The ablation experiment showed that such performance was attributed to both SRS and SHG microscopies, which increased the sensitivity of recognizing WDLPS by 16.0 and 3.6%, respectively. Furthermore, we utilized this model on RLPS margins to identify the tumor infiltration. Our method holds great potential for accurate intraoperative liposarcoma detection.

In situ brain tumor detection using a Raman spectroscopy system-results of a multicenter study

Safe and effective brain tumor surgery aims to remove tumor tissue, not non-tumoral brain. This is a challenge since tumor cells are often not visually distinguishable from peritumoral brain during surgery. To address this, we conducted a multicenter study testing whether the Sentry System could distinguish the three most common types of brain tumors from brain tissue in a label-free manner. The Sentry System is a new real time, in situ brain tumor detection device that merges Raman spectroscopy with machine learning tissue classifiers. Nine hundred and seventy-six in situ spectroscopy measurements and colocalized tissue specimens were acquired from 67 patients undergoing surgery for glioblastoma, brain metastases, or meningioma to assess tumor classification. The device achieved diagnostic accuracies of 91% for glioblastoma, 97% for brain metastases, and 96% for meningiomas. These data show that the Sentry System discriminated tumor containing tissue from non-tumoral brain in real time and prior to resection.

Breaking new ground: can artificial intelligence and machine learning transform papillary glioneuronal tumor diagnosis?

Papillary glioneuronal tumors (PGNTs), classified as Grade I by the WHO in 2016, present diagnostic challenges due to their rarity and potential for malignancy. Xiaodan Du et al.'s recent study of 36 confirmed PGNT cases provides critical insights into their imaging characteristics, revealing frequent presentation with headaches, seizures, and mass effect symptoms, predominantly located in the supratentorial region near the lateral ventricles. Lesions often appeared as mixed cystic and solid masses with septations or as cystic masses with mural nodules. Given these complexities, artificial intelligence (AI) and machine learning (ML) offer promising advancements for PGNT diagnosis. Previous studies have demonstrated AI's efficacy in diagnosing various brain tumors, utilizing deep learning and advanced imaging techniques for rapid and accurate identification. Implementing AI in PGNT diagnosis involves assembling comprehensive datasets, preprocessing data, extracting relevant features, and iteratively training models for optimal performance. Despite AI's potential, medical professionals must validate AI predictions, ensuring they complement rather than replace clinical expertise. This integration of AI and ML into PGNT diagnostics could significantly enhance preoperative accuracy, ultimately improving patient outcomes through more precise and timely interventions.

Deep learning model to differentiate Crohn's disease from intestinal tuberculosis using histopathological whole slide images from intestinal specimens

Crohn's disease (CD) and intestinal tuberculosis (ITB) share similar histopathological characteristics, and differential diagnosis can be a dilemma for pathologists. This study aimed to apply deep learning (DL) to analyze whole slide images (WSI) of surgical resection specimens to distinguish CD from ITB. Overall, 1973 WSI from 85 cases from 3 centers were obtained. The DL model was established in internal training and validated in external test cohort, evaluated by area under receiver operator characteristic curve (AUC). Diagnostic results of pathologists were compared with those of the DL model using DeLong's test. DL model had case level AUC of 0.886, 0.893 and slide level AUC of 0.954, 0.827 in training and test cohorts. Attention maps highlighted discriminative areas and top 10 features were extracted from CD and ITB. DL model's diagnostic efficiency (AUC = 0.886) was better than junior pathologists (*1 AUC = 0.701, P = 0.088; *2 AUC = 0.861, P = 0.788) and inferior to senior GI pathologists (*3 AUC = 0.910, P = 0.800; *4 AUC = 0.946, P = 0.507) in training cohort. In the test cohort, model (AUC = 0.893) outperformed senior non-GI pathologists (*5 AUC = 0.782, P = 0.327; *6 AUC = 0.821, P = 0.516). We developed a DL model for the classification of CD and ITB, improving pathological diagnosis accuracy effectively.

Lung Cancer Diagnosis on Virtual Histologically Stained Tissue Using Weakly Supervised Learning

Lung adenocarcinoma (LUAD) is the most common primary lung cancer and accounts for 40% of all lung cancer cases. The current gold standard for lung cancer analysis is based on the pathologists' interpretation of hematoxylin and eosin (H&E)-stained tissue slices viewed under a brightfield microscope or a digital slide scanner. Computational pathology using deep learning has been proposed to detect lung cancer on histology images. However, the histological staining workflow to acquire the H&E-stained images and the subsequent cancer diagnosis procedures are labor-intensive and time-consuming with tedious sample preparation steps and repetitive manual interpretation, respectively. In this work, we propose a weakly supervised learning method for LUAD classification on label-free tissue slices with virtual histological staining. The autofluorescence images of label-free tissue with histopathological information can be converted into virtual H&E-stained images by a weakly supervised deep generative model. For the downstream LUAD classification task, we trained the attention-based multiple-instance learning model with different settings on the open-source LUAD H&E-stained whole-slide images (WSIs) dataset from the Cancer Genome Atlas (TCGA). The model was validated on the 150 H&E-stained WSIs collected from patients in Queen Mary Hospital and Prince of Wales Hospital with an average area under the curve (AUC) of 0.961. The model also achieved an average AUC of 0.973 on 58 virtual H&E-stained WSIs, comparable to the results on 58 standard H&E-stained WSIs with an average AUC of 0.977. The attention heatmaps of virtual H&E-stained WSIs and ground-truth H&E-stained WSIs can indicate tumor regions of LUAD tissue slices. In conclusion, the proposed diagnostic workflow on virtual H&E-stained WSIs of label-free tissue is a rapid, cost effective, and interpretable approach to assist clinicians in postoperative pathological examinations. The method could serve as a blueprint for other label-free imaging modalities and disease contexts.

Localization of protoporphyrin IX during glioma-resection surgery via paired stimulated Raman histology and fluorescence microscopy

The most widely used fluorophore in glioma-resection surgery, 5-aminolevulinic acid (5-ALA), is thought to cause the selective accumulation of fluorescent protoporphyrin IX (PpIX) in tumour cells. Here we show that the clinical detection of PpIX can be improved via a microscope that performs paired stimulated Raman histology and two-photon excitation fluorescence microscopy (TPEF). We validated the technique in fresh tumour specimens from 115 patients with high-grade gliomas across four medical institutions. We found a weak negative correlation between tissue cellularity and the fluorescence intensity of PpIX across all imaged specimens. Semi-supervised clustering of the TPEF images revealed five distinct patterns of PpIX fluorescence, and spatial transcriptomic analyses of the imaged tissue showed that myeloid cells predominate in areas where PpIX accumulates in the intracellular space. Further analysis of external spatially resolved metabolomics, transcriptomics and RNA-sequencing datasets from glioblastoma specimens confirmed that myeloid cells preferentially accumulate and metabolize PpIX. Our findings question 5-ALA-induced fluorescence in glioma cells and show how 5-ALA and TPEF imaging can provide a window into the immune microenvironment of gliomas.

Label-free optical imaging for brain cancer assessment

Advances in label-free optical imaging offer a promising avenue for brain cancer assessment, providing high-resolution, real-time insights without the need for radiation or exogeneous agents. These cost-effective and intricately detailed techniques overcome the limitations inherent in magnetic resonance imaging (MRI), computed tomography (CT), and positron emission tomography (PET) scans by offering superior resolution and more readily accessible imaging options. This comprehensive review explores a variety of such methods, including photoacoustic imaging (PAI), optical coherence tomography (OCT), Raman imaging, and IR microscopy. It focuses on their roles in the detection, diagnosis, and management of brain tumors. By highlighting recent advances in these imaging techniques, the review aims to underscore the importance of label-free optical imaging in enhancing early detection and refining therapeutic strategies for brain cancer.

A Contemporary Approach to Intraoperative Evaluation in Neuropathology

CONTEXT.—

Although the basic principles of intraoperative diagnosis in surgical neuropathology have not changed in the last century, the last several decades have seen dramatic changes in tumor classification, terminology, molecular classification, and modalities used for intraoperative diagnosis. As many neuropathologic intraoperative diagnoses are performed by general surgical pathologists, awareness of these recent changes is important for the most accurate intraoperative diagnosis.

OBJECTIVE.—

To describe recent changes in the practice of intraoperative surgical neuropathology, with an emphasis on new entities, tumor classification, and anticipated ancillary tests, including molecular testing.

DATA SOURCES.—

The sources for this review include the fifth edition of the World Health Organization Classification of Tumours of the Central Nervous System, primary literature on intraoperative diagnosis and newly described tumor entities, and the authors' clinical experience.

CONCLUSIONS.—

A significant majority of neuropathologic diagnoses require ancillary testing, including molecular analysis, for appropriate classification. Therefore, the primary goal for any neurosurgical intraoperative diagnosis is the identification of diagnostic tissue and the preservation of the appropriate tissue for molecular testing. The intraoperative pathologist should seek to place a tumor in the most accurate diagnostic category possible, but specific diagnosis at the time of an intraoperative diagnosis is often not possible. Many entities have seen adjustments to grading criteria, including the incorporation of molecular features into grading. Awareness of these changes can help to avoid overgrading or undergrading at the time of intraoperative evaluation.

Super-resolution of biomedical volumes with 2D supervision

Volumetric biomedical microscopy has the potential to increase the diagnostic information extracted from clinical tissue specimens and improve the diagnostic accuracy of both human pathologists and computational pathology models. Unfortunately, barriers to integrating 3-dimensional (3D) volumetric microscopy into clinical medicine include long imaging times, poor depth/z-axis resolution, and an insufficient amount of high-quality volumetric data. Leveraging the abundance of high-resolution 2D microscopy data, we introduce masked slice diffusion for super-resolution (MSDSR), which exploits the inherent equivalence in the data-generating distribution across all spatial dimensions of biological specimens. This intrinsic characteristic allows for super-resolution models trained on high-resolution images from one plane (e.g., XY) to effectively generalize to others (XZ, YZ), overcoming the traditional dependency on orientation. We focus on the application of MSDSR to stimulated Raman histology (SRH), an optical imaging modality for biological specimen analysis and intraoperative diagnosis, characterized by its rapid acquisition of high-resolution 2D images but slow and costly optical z-sectioning. To evaluate MSDSR's efficacy, we introduce a new performance metric, SliceFID, and demonstrate MSDSR's superior performance over baseline models through extensive evaluations. Our findings reveal that MSDSR not only significantly enhances the quality and resolution of 3D volumetric data, but also addresses major obstacles hindering the broader application of 3D volumetric microscopy in clinical diagnostics and biomedical research.

Fast Real-Time Brain Tumor Detection Based on Stimulated Raman Histology and Self-Supervised Deep Learning Model

In intraoperative brain cancer procedures, real-time diagnosis is essential for ensuring safe and effective care. The prevailing workflow, which relies on histological staining with hematoxylin and eosin (H&E) for tissue processing, is resource-intensive, time-consuming, and requires considerable labor. Recently, an innovative approach combining stimulated Raman histology (SRH) and deep convolutional neural networks (CNN) has emerged, creating a new avenue for real-time cancer diagnosis during surgery. While this approach exhibits potential, there exists an opportunity for refinement in the domain of feature extraction. In this study, we employ coherent Raman scattering imaging method and a self-supervised deep learning model (VQVAE2) to enhance the speed of SRH image acquisition and feature representation, thereby enhancing the capability of automated real-time bedside diagnosis. Specifically, we propose the VQSRS network, which integrates vector quantization with a proxy task based on patch annotation for analysis of brain tumor subtypes. Training on images collected from the SRS microscopy system, our VQSRS demonstrates a significant speed enhancement over traditional techniques (e.g., 20-30 min). Comparative studies in dimensionality reduction clustering confirm the diagnostic capacity of VQSRS rivals that of CNN. By learning a hierarchical structure of recognizable histological features, VQSRS classifies major tissue pathological categories in brain tumors. Additionally, an external semantic segmentation method is applied for identifying tumor-infiltrated regions in SRH images. Collectively, these findings indicate that this automated real-time prediction technique holds the potential to streamline intraoperative cancer diagnosis, providing assistance to pathologists in simplifying the process.

Multibranch CNN With MLP-Mixer-Based Feature Exploration for High-Performance Disease Diagnosis

Deep learning-based diagnosis is becoming an indispensable part of modern healthcare. For high-performance diagnosis, the optimal design of deep neural networks (DNNs) is a prerequisite. Despite its success in image analysis, existing supervised DNNs based on convolutional layers often suffer from their rudimentary feature exploration ability caused by the limited receptive field and biased feature extraction of conventional convolutional neural networks (CNNs), which compromises the network performance. Here, we propose a novel feature exploration network named manifold embedded multilayer perceptron (MLP) mixer (ME-Mixer), which utilizes both supervised and unsupervised features for disease diagnosis. In the proposed approach, a manifold embedding network is employed to extract class-discriminative features; then, two MLP-Mixer-based feature projectors are adopted to encode the extracted features with the global reception field. Our ME-Mixer network is quite general and can be added as a plugin to any existing CNN. Comprehensive evaluations on two medical datasets are performed. The results demonstrate that their approach greatly enhances the classification accuracy in comparison with different configurations of DNNs with acceptable computational complexity.

Ultra-Rapid Droplet Digital PCR Enables Intraoperative Tumor Quantification

The diagnosis and treatment of tumors often depends on molecular-genetic data. However, rapid and iterative access to molecular data is not currently feasible during surgery, complicating intraoperative diagnosis and precluding measurement of tumor cell burdens at surgical margins to guide resections. To address this gap, we developed Ultra-Rapid droplet digital PCR (UR-ddPCR), which can be completed in 15 minutes from tissue to result with an accuracy comparable to standard ddPCR. We demonstrate UR-ddPCR assays for the IDH1 R132H and BRAF V600E clonal mutations that are present in many low-grade gliomas and melanomas, respectively. We illustrate the clinical feasibility of UR-ddPCR by performing it intraoperatively for 13 glioma cases. We further combine UR-ddPCR measurements with UR-stimulated Raman histology intraoperatively to estimate tumor cell densities in addition to tumor cell percentages. We anticipate that UR-ddPCR, along with future refinements in assay instrumentation, will enable novel point-of-care diagnostics and the development of molecularly-guided surgeries that improve clinical outcomes.

Practical Application of Deep Learning in Diagnostic Neuropathology-Reimagining a Histological Asset in the Era of Precision Medicine

In the past few decades, neuropathology has experienced several paradigm shifts with the introduction of new technologies. Deep learning, a rapidly progressing subfield of machine learning, seems to be the next innovation to alter the diagnostic workflow. In this review, we will explore the recent changes in the field of neuropathology and how this has led to an increased focus on molecular features in diagnosis and prognosis. Then, we will examine the work carried out to train deep learning models for various diagnostic tasks in neuropathology, as well as the machine learning frameworks they used. Focus will be given to both the challenges and successes highlighted therein, as well as what these trends may tell us about future roadblocks in the widespread adoption of this new technology. Finally, we will touch on recent trends in deep learning, as applied to digital pathology more generally, and what this may tell us about the future of deep learning applications in neuropathology.

Optical Methods for Brain Tumor Detection: A Systematic Review

Background: In brain tumor surgery, maximal tumor resection is typically desired. This is complicated by infiltrative tumor cells which cannot be visually distinguished from healthy brain tissue. Optical methods are an emerging field that can potentially revolutionize brain tumor surgery through intraoperative differentiation between healthy and tumor tissues. Methods: This study aimed to systematically explore and summarize the existing literature on the use of Raman Spectroscopy (RS), Hyperspectral Imaging (HSI), Optical Coherence Tomography (OCT), and Diffuse Reflectance Spectroscopy (DRS) for brain tumor detection. MEDLINE, Embase, and Web of Science were searched for studies evaluating the accuracy of these systems for brain tumor detection. Outcome measures included accuracy, sensitivity, and specificity. Results: In total, 44 studies were included, covering a range of tumor types and technologies. Accuracy metrics in the studies ranged between 54 and 100% for RS, 69 and 99% for HSI, 82 and 99% for OCT, and 42 and 100% for DRS. Conclusions: This review provides insightful evidence on the use of optical methods in distinguishing tumor from healthy brain tissue.

Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions

Artificial intelligence (AI) in oncology is advancing beyond algorithm development to integration into clinical practice. This review describes the current state of the field, with a specific focus on clinical integration. AI applications are structured according to cancer type and clinical domain, focusing on the four most common cancers and tasks of detection, diagnosis, and treatment. These applications encompass various data modalities, including imaging, genomics, and medical records. We conclude with a summary of existing challenges, evolving solutions, and potential future directions for the field.

SIGNIFICANCE

AI is increasingly being applied to all aspects of oncology, where several applications are maturing beyond research and development to direct clinical integration. This review summarizes the current state of the field through the lens of clinical translation along the clinical care continuum. Emerging areas are also highlighted, along with common challenges, evolving solutions, and potential future directions for the field.

Virtual Staining of Nonfixed Tissue Histology

Surgical pathology workflow involves multiple labor-intensive steps, such as tissue removal, fixation, embedding, sectioning, staining, and microscopic examination. This process is time-consuming and costly and requires skilled technicians. In certain clinical scenarios, such as intraoperative consultations, there is a need for faster histologic evaluation to provide real-time surgical guidance. Currently, frozen section techniques involving hematoxylin and eosin (H&E) staining are used for intraoperative pathology consultations. However, these techniques have limitations, including a turnaround time of 20 to 30 minutes, staining artifacts, and potential tissue loss, negatively impacting accurate diagnosis. To address these challenges, researchers are exploring alternative optical imaging modalities for rapid microscopic tissue imaging. These modalities differ in optical characteristics, tissue preparation requirements, imaging equipment, and output image quality and format. Some of these imaging methods have been combined with computational algorithms to generate H&E-like images, which could greatly facilitate their adoption by pathologists. Here, we provide a comprehensive, organ-specific review of the latest advancements in emerging imaging modalities applied to nonfixed human tissue. We focused on studies that generated H&E-like images evaluated by pathologists. By presenting up-to-date research progress and clinical utility, this review serves as a valuable resource for scholars and clinicians, covering some of the major technical developments in this rapidly evolving field. It also offers insights into the potential benefits and drawbacks of alternative imaging modalities and their implications for improving patient care.

Assessment of Drug Activities against Giardia Using Hyperspectral Raman Microscopy

This study demonstrates the capability of Raman microscopy for detecting structural differences in Giardia cells exposed to different drugs and incubation times. While metronidazole (MTZ) visibly affects the cells by inducing extracellular vesicle releases of toxic iron intermediates and modified triple-bond moieties, oseltamivir (OSM) alters the phenylalanine and lipid structures. Modifications in the heme protein environment and the transformation of iron from ferric to ferrous observed for both drug treatments are more notable for MTZ. Different contents and amounts of vesicle excretion are detected for 24 h or 48 h with MTZ incubation. At a shorter drug exposure, releases of altered proteins, glycogen, and phospholipids dominate. Agglomerates of transformed iron complexes from heme proteins and multiple-bond moieties prevail at 48 h of treatment. No such vesicle releases are present in the case of OSM usage. Drug incorporations into the cells and their impact on the plasma membrane and the dynamics of lipid raft confirmed by confocal fluorescence microscopy reveal a more destructive extent by OSM, corroborating the Raman results. Raman microscopy provides a broader understanding of the multifaceted factors and mechanisms responsible for giardiasis treatment or drug resistance by enabling a label-free, simultaneous monitoring of structural changes at the cellular and molecular levels.

Raman difference spectroscopy and U-Net convolutional neural network for molecular analysis of cutaneous neurofibroma

In Neurofibromatosis type 1 (NF1), peripheral nerve sheaths tumors are common, with cutaneous neurofibromas resulting in significant aesthetic, painful and functional problems requiring surgical removal. To date, determination of adequate surgical resection margins-complete tumor removal while attempting to preserve viable tissue-remains largely subjective. Thus, residual tumor extension beyond surgical margins or recurrence of the disease may frequently be observed. Here, we introduce Shifted-Excitation Raman Spectroscopy in combination with deep neural networks for the future perspective of objective, real-time diagnosis, and guided surgical ablation. The obtained results are validated through established histological methods. In this study, we evaluated the discrimination between cutaneous neurofibroma (n = 9) and adjacent physiological tissues (n = 25) in 34 surgical pathological specimens ex vivo at a total of 82 distinct measurement loci. Based on a convolutional neural network (U-Net), the mean raw Raman spectra (n = 8,200) were processed and refined, and afterwards the spectral peaks were assigned to their respective molecular origin. Principal component and linear discriminant analysis was used to discriminate cutaneous neurofibromas from physiological tissues with a sensitivity of 100%, specificity of 97.3%, and overall classification accuracy of 97.6%. The results enable the presented optical, non-invasive technique in combination with artificial intelligence as a promising candidate to ameliorate both, diagnosis and treatment of patients affected by cutaneous neurofibroma and NF1.

Current Landscape of Advanced Imaging Tools for Pathology Diagnostics

Histopathology relies on century-old workflows of formalin fixation, paraffin embedding, sectioning, and staining tissue specimens on glass slides. Despite being robust, this conventional process is slow, labor-intensive, and limited to providing two-dimensional views. Emerging technologies promise to enhance and accelerate histopathology. Slide-free microscopy allows rapid imaging of fresh, unsectioned specimens, overcoming slide preparation delays. Methods such as fluorescence confocal microscopy, multiphoton microscopy, along with more recent innovations including microscopy with UV surface excitation and fluorescence-imitating brightfield imaging can generate images resembling conventional histology directly from the surface of tissue specimens. Slide-free microscopy enable applications such as rapid intraoperative margin assessment and, with appropriate technology, three-dimensional histopathology. Multiomics profiling techniques, including imaging mass spectrometry and Raman spectroscopy, provide highly multiplexed molecular maps of tissues, although clinical translation remains challenging. Artificial intelligence is aiding the adoption of new imaging modalities via virtual staining, which converts methods such as slide-free microscopy into synthetic brightfield-like or even molecularly informed images. Although not yet commonplace, these emerging technologies collectively demonstrate the potential to modernize histopathology. Artificial intelligence-assisted workflows will ease the transition to new imaging modalities. With further validation, these advances may transform the century-old conventional histopathology pipeline to better serve 21st-century medicine. This review provides an overview of these enabling technology platforms and discusses their potential impact.

[Intraoperative stimulated Raman histology for personalized brain tumor surgery]

BACKGROUND

In brain tumor surgery a personalized surgical approach is crucial to achieve a maximum safe tumor resection. The extent of resection decisively depends on the histological diagnosis. Stimulated Raman histology (SRH), a fiber laser-based optical imaging method, offers the possibility for evaluation of an intraoperative diagnosis in a few minutes.

OBJECTIVE

To provide an overview on the applications of SRH in neurosurgery and transference of the technique to other surgical disciplines.

METHODS

Description of the technique and review of the current literature on SRH.

RESULTS

The SRH technique was successfully used in multiple neuro-oncological tumor entities. Initial pilot projects showed the potential for analysis of extracranial tumors.

CONCLUSION

The use of SRH provides a near real-time diagnosis with high diagnostic accuracy and provides further developmental potential to improve personalized tumor surgery.

Machine Learning-Assisted Classification of Paraffin-Embedded Brain Tumors with Raman Spectroscopy

Raman spectroscopy (RS) has demonstrated its utility in neurooncological diagnostics, spanning from intraoperative tumor detection to the analysis of tissue samples peri- and postoperatively. In this study, we employed Raman spectroscopy (RS) to monitor alterations in the molecular vibrational characteristics of a broad range of formalin-fixed, paraffin-embedded (FFPE) intracranial neoplasms (including primary brain tumors and meningiomas, as well as brain metastases) and considered specific challenges when employing RS on FFPE tissue during the routine neuropathological workflow. We spectroscopically measured 82 intracranial neoplasms on CaF2 slides (in total, 679 individual measurements) and set up a machine learning framework to classify spectral characteristics by splitting our data into training cohorts and external validation cohorts. The effectiveness of our machine learning algorithms was assessed by using common performance metrics such as AUROC and AUPR values. With our trained random forest algorithms, we distinguished among various types of gliomas and identified the primary origin in cases of brain metastases. Moreover, we spectroscopically diagnosed tumor types by using biopsy fragments of pure necrotic tissue, a task unattainable through conventional light microscopy. In order to address misclassifications and enhance the assessment of our models, we sought out significant Raman bands suitable for tumor identification. Through the validation phase, we affirmed a considerable complexity within the spectroscopic data, potentially arising not only from the biological tissue subjected to a rigorous chemical procedure but also from residual components of the fixation and paraffin-embedding process. The present study demonstrates not only the potential applications but also the constraints of RS as a diagnostic tool in neuropathology, considering the challenges associated with conducting vibrational spectroscopic analysis on formalin-fixed, paraffin-embedded (FFPE) tissue.

Development and prospective validation of an artificial intelligence-based smartphone app for rapid intraoperative pituitary adenoma identification

BACKGROUND

Intraoperative pathology consultation plays a crucial role in tumor surgery. The ability to accurately and rapidly distinguish tumor from normal tissue can greatly impact intraoperative surgical oncology management. However, this is dependent on the availability of a specialized pathologist for a reliable diagnosis. We developed and prospectively validated an artificial intelligence-based smartphone app capable of differentiating between pituitary adenoma and normal pituitary gland using stimulated Raman histology, almost instantly.

METHODS

The study consisted of three parts. After data collection (part 1) and development of a deep learning-based smartphone app (part 2), we conducted a prospective study that included 40 consecutive patients with 194 samples to evaluate the app in real-time in a surgical setting (part 3). The smartphone app's sensitivity, specificity, positive predictive value, and negative predictive value were evaluated by comparing the diagnosis rendered by the app to the ground-truth diagnosis set by a neuropathologist.

RESULTS

The app exhibits a sensitivity of 96.1% (95% CI: 89.9-99.0%), specificity of 92.7% (95% CI: 74-99.3%), positive predictive value of 98% (95% CI: 92.2-99.8%), and negative predictive value of 86.4% (95% CI: 66.2-96.8%). An external validation of the smartphone app on 40 different adenoma tumors and a total of 191 scanned SRH specimens from a public database shows a sensitivity of 93.7% (95% CI: 89.3-96.7%).

CONCLUSIONS

The app can be readily expanded and repurposed to work on different types of tumors and optical images. Rapid recognition of normal versus tumor tissue during surgery may contribute to improved intraoperative surgical management and oncologic outcomes. In addition to the accelerated pathological assessments during surgery, this platform can be of great benefit in community hospitals and developing countries, where immediate access to a specialized pathologist during surgery is limited.

Noninvasive Nonlinear Optical Computational Histology

Cancer remains a global health challenge, demanding early detection and accurate diagnosis for improved patient outcomes. An intelligent paradigm is introduced that elevates label-free nonlinear optical imaging with contrastive patch-wise learning, yielding stain-free nonlinear optical computational histology (NOCH). NOCH enables swift, precise diagnostic analysis of fresh tissues, reducing patient anxiety and healthcare costs. Nonlinear modalities are evaluated, including stimulated Raman scattering and multiphoton imaging, for their ability to enhance tumor microenvironment sensitivity, pathological analysis, and cancer examination. Quantitative analysis confirmed that NOCH images accurately reproduce nuclear morphometric features across different cancer stages. Key diagnostic features, such as nuclear morphology, size, and nuclear-cytoplasmic contrast, are well preserved. NOCH models also demonstrate promising generalization when applied to other pathological tissues. The study unites label-free nonlinear optical imaging with histopathology using contrastive learning to establish stain-free computational histology. NOCH provides a rapid, non-invasive, and precise approach to surgical pathology, holding immense potential for revolutionizing cancer diagnosis and surgical interventions.

Computational Assessment of Spectral Heterogeneity within Fresh Glioblastoma Tissue Using Raman Spectroscopy and Machine Learning Algorithms

Understanding and classifying inherent tumor heterogeneity is a multimodal approach, which can be undertaken at the genetic, biochemical, or morphological level, among others. Optical spectral methods such as Raman spectroscopy aim at rapid and non-destructive tissue analysis, where each spectrum generated reflects the individual molecular composition of an examined spot within a (heterogenous) tissue sample. Using a combination of supervised and unsupervised machine learning methods as well as a solid database of Raman spectra of native glioblastoma samples, we succeed not only in distinguishing explicit tumor areas-vital tumor tissue and necrotic tumor tissue can correctly be predicted with an accuracy of 76%-but also in determining and classifying different spectral entities within the histomorphologically distinct class of vital tumor tissue. Measurements of non-pathological, autoptic brain tissue hereby serve as a healthy control since their respective spectroscopic properties form an individual and reproducible cluster within the spectral heterogeneity of a vital tumor sample. The demonstrated decipherment of a spectral glioblastoma heterogeneity will be valuable, especially in the field of spectroscopically guided surgery to delineate tumor margins and to assist resection control.

Brain tumour detection from magnetic resonance imaging using convolutional neural networks

Introduction

The aim of this work is to detect and classify brain tumours using computational intelligence techniques on magnetic resonance imaging (MRI) images.

Material and methods

A dataset of 3264 MRI brain images consisting of 4 categories: unspecified glioma, meningioma, pituitary, and healthy brain, was used in this study. Twelve convolutional neural networks (GoogleNet, MobileNetV2, Xception, DesNet-BC, ResNet 50, SqueezeNet, ShuffleNet, VGG-16, AlexNet, Enet, EfficientB0, and MobileNetV2 with meta pseudo-labels) were used to classify gliomas, meningiomas, pituitary tumours, and healthy brains to find the most appropriate model. The experiments included image preprocessing and hyperparameter tuning. The performance of each neural network was evaluated based on accuracy, precision, recall, and F-measure for each type of brain tumour.

Results

The experimental results show that the MobileNetV2 convolutional neural network (CNN) model was able to diagnose brain tumours with 99% accuracy, 98% recall, and 99% F1 score. On the other hand, the validation data analysis shows that the CNN model GoogleNet has the highest accuracy (97%) among CNNs and seems to be the best choice for brain tumour classification.

Conclusions

The results of this work highlight the importance of artificial intelligence and machine learning for brain tumour prediction. Furthermore, this study achieved the highest accuracy in brain tumour classification to date, and it is also the only study to compare the performance of so many neural networks simultaneously.

AI-Based Detection of Oral Squamous Cell Carcinoma with Raman Histology

Stimulated Raman Histology (SRH) employs the stimulated Raman scattering (SRS) of photons at biomolecules in tissue samples to generate histological images. Subsequent pathological analysis allows for an intraoperative evaluation without the need for sectioning and staining. The objective of this study was to investigate a deep learning-based classification of oral squamous cell carcinoma (OSCC) and the sub-classification of non-malignant tissue types, as well as to compare the performances of the classifier between SRS and SRH images. Raman shifts were measured at wavenumbers k1 = 2845 cm-1 and k2 = 2930 cm-1. SRS images were transformed into SRH images resembling traditional H&E-stained frozen sections. The annotation of 6 tissue types was performed on images obtained from 80 tissue samples from eight OSCC patients. A VGG19-based convolutional neural network was then trained on 64 SRS images (and corresponding SRH images) and tested on 16. A balanced accuracy of 0.90 (0.87 for SRH images) and F1-scores of 0.91 (0.91 for SRH) for stroma, 0.98 (0.96 for SRH) for adipose tissue, 0.90 (0.87 for SRH) for squamous epithelium, 0.92 (0.76 for SRH) for muscle, 0.87 (0.90 for SRH) for glandular tissue, and 0.88 (0.87 for SRH) for tumor were achieved. The results of this study demonstrate the suitability of deep learning for the intraoperative identification of tissue types directly on SRS and SRH images.

High-content stimulated Raman histology of human breast cancer

Histological examination is crucial for cancer diagnosis, however, the labor-intensive sample preparation involved in the histology impedes the speed of diagnosis. Recently developed two-color stimulated Raman histology could bypass the complex tissue processing to generates result close to hematoxylin and eosin staining, which is one of the golden standards in cancer histology. Yet, the underlying chemical features are not revealed in two-color stimulated Raman histology, compromising the effectiveness of prognostic stratification. Here, we present a high-content stimulated Raman histology (HC-SRH) platform that provides both morphological and chemical information for cancer diagnosis based on un-stained breast tissues. Methods: By utilizing both hyperspectral SRS imaging in the C-H vibration window and sparsity-penalized unmixing of overlapped spectral profiles, HC-SRH enabled high-content chemical mapping of saturated lipids, unsaturated lipids, cellular protein, extracellular matrix (ECM), and water. Spectral selective sampling was further implemented to boost the speed of HC-SRH. To show the potential for clinical use, HC-SRH using a compact fiber laser-based stimulated Raman microscope was demonstrated. Harnessing the wide and rapid tuning capability of the fiber laser, both C-H and fingerprint vibration windows were accessed. Results: HC-SRH successfully mapped unsaturated lipids, cellular protein, extracellular matrix, saturated lipid, and water in breast tissue. With these five chemical maps, HC-SRH provided distinct contrast for tissue components including duct, stroma, fat cell, necrosis, and vessel. With selective spectral sampling, the speed of HC-SRH was improved by one order of magnitude. The fiber-laser-based HC-SRH produced the same image quality in the C-H window as the state-of-the-art solid laser. In the fingerprint window, nucleic acid and solid-state ester contrast was demonstrated. Conclusions: HC-SRH provides both morphological and chemical information of tissue in a label-free manner. The chemical information detected is beyond the reach of traditional hematoxylin and eosin staining and heralds the potential of HC-SRH for biomarker discovery.

Noise learning of instruments for high-contrast, high-resolution and fast hyperspectral microscopy and nanoscopy

The low scattering efficiency of Raman scattering makes it challenging to simultaneously achieve good signal-to-noise ratio (SNR), high imaging speed, and adequate spatial and spectral resolutions. Here, we report a noise learning (NL) approach that estimates the intrinsic noise distribution of each instrument by statistically learning the noise in the pixel-spatial frequency domain. The estimated noise is then removed from the noisy spectra. This enhances the SNR by ca. 10 folds, and suppresses the mean-square error by almost 150 folds. NL allows us to improve the positioning accuracy and spatial resolution and largely eliminates the impact of thermal drift on tip-enhanced Raman spectroscopic nanoimaging. NL is also applicable to enhance SNR in fluorescence and photoluminescence imaging. Our method manages the ground truth spectra and the instrumental noise simultaneously within the training dataset, which bypasses the tedious labelling of huge dataset required in conventional deep learning, potentially shifting deep learning from sample-dependent to instrument-dependent.

Rapid single-particle chemical imaging of nanoplastics by SRS microscopy

Plastics are now omnipresent in our daily lives. The existence of microplastics (1 µm to 5 mm in length) and possibly even nanoplastics (<1 μm) has recently raised health concerns. In particular, nanoplastics are believed to be more toxic since their smaller size renders them much more amenable, compared to microplastics, to enter the human body. However, detecting nanoplastics imposes tremendous analytical challenges on both the nano-level sensitivity and the plastic-identifying specificity, leading to a knowledge gap in this mysterious nanoworld surrounding us. To address these challenges, we developed a hyperspectral stimulated Raman scattering (SRS) imaging platform with an automated plastic identification algorithm that allows micro-nano plastic analysis at the single-particle level with high chemical specificity and throughput. We first validated the sensitivity enhancement of the narrow band of SRS to enable high-speed single nanoplastic detection below 100 nm. We then devised a data-driven spectral matching algorithm to address spectral identification challenges imposed by sensitive narrow-band hyperspectral imaging and achieve robust determination of common plastic polymers. With the established technique, we studied the micro-nano plastics from bottled water as a model system. We successfully detected and identified nanoplastics from major plastic types. Micro-nano plastics concentrations were estimated to be about 2.4 ± 1.3 × 105 particles per liter of bottled water, about 90% of which are nanoplastics. This is orders of magnitude more than the microplastic abundance reported previously in bottled water. High-throughput single-particle counting revealed extraordinary particle heterogeneity and nonorthogonality between plastic composition and morphologies; the resulting multidimensional profiling sheds light on the science of nanoplastics.

Intraoperative margin assessment for basal cell carcinoma with deep learning and histologic tumor mapping to surgical site

Successful treatment of solid cancers relies on complete surgical excision of the tumor either for definitive treatment or before adjuvant therapy. Intraoperative and postoperative radial sectioning, the most common form of margin assessment, can lead to incomplete excision and increase the risk of recurrence and repeat procedures. Mohs Micrographic Surgery is associated with complete removal of basal cell and squamous cell carcinoma through real-time margin assessment of 100% of the peripheral and deep margins. Real-time assessment in many tumor types is constrained by tissue size, complexity, and specimen processing / assessment time during general anesthesia. We developed an artificial intelligence platform to reduce the tissue preprocessing and histological assessment time through automated grossing recommendations, mapping and orientation of tumor to the surgical specimen. Using basal cell carcinoma as a model system, results demonstrate that this approach can address surgical laboratory efficiency bottlenecks for rapid and complete intraoperative margin assessment.

A perspective on computer vision in biosensing

Computer vision has become a powerful tool in the field of biosensing, aiding in the development of innovative and precise systems for the analysis and interpretation of biological data. This interdisciplinary approach harnesses the capabilities of computer vision algorithms and techniques to extract valuable information from various biosensing applications, including medical diagnostics, environmental monitoring, and food health. Despite years of development, there is still significant room for improvement in this area. In this perspective, we outline how computer vision is applied to raw sensor data in biosensors and its advantages to biosensing applications. We then discuss ongoing research and developments in the field and subsequently explore the challenges and opportunities that computer vision faces in biosensor applications. We also suggest directions for future work, ultimately underscoring the significant impact of computer vision on advancing biosensing technologies and their applications.

Adaptive Cancer Therapy in the Age of Generative Artificial Intelligence

Therapeutic resistance is a major challenge facing the design of effective cancer treatments. Adaptive cancer therapy is in principle the most viable approach to manage cancer's adaptive dynamics through drug combinations with dose timing and modulation. However, there are numerous open issues facing the clinical success of adaptive therapy. Chief among these issues is the feasibility of real-time predictions of treatment response which represent a bedrock requirement of adaptive therapy. Generative artificial intelligence has the potential to learn prediction models of treatment response from clinical, molecular, and radiomics data about patients and their treatments. The article explores this potential through a proposed integration model of Generative Pre-Trained Transformers (GPTs) in a closed loop with adaptive treatments to predict the trajectories of disease progression. The conceptual model and the challenges facing its realization are discussed in the broader context of artificial intelligence integration in oncology.