Multimodal Ensemble Fusion Deep Learning Using Histopathological Images and Clinical Data for Glioma Subtype Classification

Glioma is the most common malignant tumor of the central nervous system, and diffuse Glioma is classified as grades II-IV by world health organization (WHO). In the the cancer genome atlas (TCGA) Glioma dataset, grade II and III Gliomas are classified as low-grade glioma (LGG), and grade IV Gliomas as glioblastoma multiforme (GBM). In clinical practice, the survival and treatment process with Glioma patients depends on properly diagnosing the subtype. With this background, there has been much research on Glioma over the years. Among these researches, the origin and evolution of whole slide images (WSIs) have led to many attempts to support diagnosis by image analysis. On the other hand, due to the disease complexities of Glioma patients, multimodal analysis using various types of data rather than a single data set has been attracting attention. In our proposed method, multiple deep learning models are used to extract features from histopathology images, and the features of the obtained images are concatenated with those of the clinical data in a fusion approach. Then, we perform patch-level classification by machine learning (ML) using the concatenated features. Based on the performances of the deep learning models, we ensemble feature sets from top three models and perform further classifications. In the experiments with our proposed ensemble fusion AI (EFAI) approach using WSIs and clinical data of diffuse Glioma patients on TCGA dataset, the classification accuracy of the proposed multimodal ensemble fusion method is 0.936 with an area under the curve (AUC) value of 0.967 when tested on a balanced dataset of 240 GBM, 240 LGG patients. On an imbalanced dataset of 141 GBM, 242 LGG patients the proposed method obtained the accuracy of 0.936 and AUC of 0.967. Our proposed ensemble fusion approach significantly outperforms the classification using only histopathology images alone with deep learning models. Therefore, our approach can be used to support the diagnosis of Glioma patients and can lead to better diagnosis.

Brain tumour histopathology through the lens of deep learning: A systematic review

PROBLEM

Machine learning (ML)/Deep learning (DL) techniques have been evolving to solve more complex diseases, but it has been used relatively little in Glioblastoma (GBM) histopathological studies, which could benefit greatly due to the disease's complex pathogenesis.

AIM

Conduct a systematic review to investigate how ML/DL techniques have influenced the progression of brain tumour histopathological research, particularly in GBM.

METHODS

54 eligible studies were collected from the PubMed and ScienceDirect databases, and their information about the types of brain tumour/s used, types of -omics data used with histopathological data, origins of the data, types of ML/DL and its training and evaluation methodologies, and the ML/DL task it was set to perform in the study were extracted to inform us of trends in GBM-related ML/DL-based research.

RESULTS

Only 8 GBM-related studies in the eligible utilised ML/DL methodologies to gain deeper insights into GBM pathogenesis by contextualising histological data with -omics data. However, we report that these studies have been published more recently. The most popular ML/DL models used in GBM-related research are the SVM classifier and ResNet-based CNN architecture. Still, a considerable number of studies failed to state training and evaluative methodologies clearly.

CONCLUSION

There is a growing trend towards using ML/DL approaches to uncover relationships between biological and histopathological data to bring new insights into GBM, thus pushing GBM research forward. Much work still needs to be done to properly report the ML/DL methodologies to showcase the models' robustness and generalizability and ensure the models are reproducible.

Context aware machine learning techniques for brain tumor classification and detection - A review

Background

Machine learning has tremendous potential in acute medical care, particularly in the field of precise medical diagnosis, prediction, and classification of brain tumors. Malignant gliomas, due to their aggressive growth and dismal prognosis, stand out among various brain tumor types. Recent advancements in understanding the genetic abnormalities that underlie these tumors have shed light on their histo-pathological and biological characteristics, which support in better classification and prognosis.

Objectives

This review aims to predict gene alterations and establish structured correlations among various tumor types, extending the prediction of genetic mutations and structures using the latest machine learning techniques. Specifically, it focuses on multi-modalities of Magnetic Resonance Imaging (MRI) and histopathology, utilizing Convolutional Neural Networks (CNN) for image processing and analysis.

Methods

The review encompasses the most recent developments in MRI, and histology image processing methods across multiple tumor classes, including Glioma, Meningioma, Pituitary, Oligodendroglioma, and Astrocytoma. It identifies challenges in tumor classification, segmentation, datasets, and modalities, employing various neural network architectures. A competitive analysis assesses the performance of CNN. Furthermore it also implies K-MEANS clustering to predict Genetic structure, Genes Clusters prediction and Molecular Alteration of various types and grades of tumors e.g. Glioma, Meningioma, Pituitary, Oligodendroglioma, and Astrocytoma.

Results

CNN and KNN structures, with their ability to extract highlights in image-based information, prove effective in tumor classification and segmentation, surmounting challenges in image analysis. Competitive analysis reveals that CNN and outperform others algorithms on publicly available datasets, suggesting their potential for precise tumor diagnosis and treatment planning.

Conclusion

Machine learning, especially through CNN and SVM algorithms, demonstrates significant potential in the accurate diagnosis and classification of brain tumors based on imaging and histo-pathological data. Further advancements in this area hold promise for improving the accuracy and efficiency of intra-operative tumor diagnosis and treatment.

Improved lung nodule segmentation with a squeeze excitation dilated attention based residual UNet

The diverse types and sizes, proximity to non-nodule structures, identical shape characteristics, and varying sizes of nodules make them challenging for segmentation methods. Although many efforts have been made in automatic lung nodule segmentation, most of them have not sufficiently addressed the challenges related to the type and size of nodules, such as juxta-pleural and juxta-vascular nodules. The current research introduces a Squeeze-Excitation Dilated Attention-based Residual U-Net (SEDARU-Net) with a robust intensity normalization technique to address the challenges related to different types and sizes of lung nodules and to achieve an improved lung nodule segmentation. After preprocessing the images with the intensity normalization method and extracting the Regions of Interest by YOLOv3, they are fed into the SEDARU-Net with dilated convolutions in the encoder part. Then, the extracted features are given to the decoder part, which involves transposed convolutions, Squeeze-Excitation Dilated Residual blocks, and skip connections equipped with an Attention Gate, to decode the feature maps and construct the segmentation mask. The proposed model was evaluated using the publicly available Lung Nodule Analysis 2016 (LUNA16) dataset, achieving a Dice Similarity Coefficient of 97.86%, IoU of 96.40%, sensitivity of 96.54%, and precision of 98.84%. Finally, it was shown that each added component to the U-Net's structure and the intensity normalization technique increased the Dice Similarity Coefficient by more than 2%. The proposed method suggests a potential clinical tool to address challenges related to the segmentation of lung nodules with different types located in the proximity of non-nodule structures.

Grade 4 astrocytoma vs. grade 4 glioblastoma: is there any clue in H&E?

Objective: Gliomas are the most common primary tumors of the central nervous system. The fifth edition of the World Health Organization (WHO) Classification of Tumors of the CNS identifies IDH mutant astrocytomas grade 4 and IDH wild type glioblastomas grade 4 as distinct entities. This study aimed to identify morphological indicators that could predict IDH mutation status in grade 4 diffuse astrocytomas and grade 4 glioblastomas among fifty patients from two groups.

Methods: Hematoxylin and eosin (H&E)-stained tumor slides were scanned using a digital scanner and further histopathological examinations were performed on digital images, with additional calculations and measurements.

Results: The study showed that, IDH-wildtype glioblastomas and IDH-mutant grade 4 astrocytomas exhibit unique morphological features, particularly in relation to levels of necrosis, microvessel density, and the presence of "C" or "Ring" shape giant cells.

Conclusion: Despite advancements in genomic biomarker technology, histology remains an essential tool for predicting patient outcomes. Therefore, pathologists must continue to investigate and document the morphological implications of molecular changes in CNS tumors.

Adaptive Compression and Reconstruction for Multidimensional Medical Image Data: A Hybrid Algorithm for Enhanced Image Quality

Spatial regions within images typically hold greater priority over adjacent areas, especially in the context of medical images (MI) where minute details can have significant clinical implications. This research addresses the challenge of compressing medical image dimensions without compromising critical information by proposing an adaptive compression algorithm. The algorithm integrates a modified image enhancement module, clustering-based segmentation, and a variety of lossless and lossy compression techniques. Edge enhancement contrast limited adaptive histogram equalization (EE-CLAHE) and 2D adaptive anisotropic diffusion filter are employed to enhance and denoise the images, followed by adaptive expectation maximization clustering (AEMC) for segmentation into regions of interest (ROI) and non-ROI. The clustering process is optimized utilizing fuzzy c-means (FCM) and Otsu thresholding. Subsequently, distinct compression schemes are applied to ROI and non-ROI regions, such as Coiflet + Haar, Coiflet + Daubecheis, modified SPIHT Huffman, EZW, and SPIHT algorithms, to ensure effective storage and transmission while preserving diagnostic details. Experimental results demonstrate that the combination of the modified SPIHT Huffman algorithm for ROI and EZW for non-ROI yields superior reconstruction quality across various measures, enabling comprehensive analysis of multi-dimensional images from MRI, CT, and X-ray modalities.

Deep learning-based IDH1 gene mutation prediction using histopathological imaging and clinical data

In the field of histopathology, many studies on the classification of whole slide images (WSIs) using artificial intelligence (AI) technology have been reported. We have studied the disease progression assessment of glioma. Adult-type diffuse gliomas, a type of brain tumor, are classified into astrocytoma, oligodendroglioma, and glioblastoma. Astrocytoma and oligodendroglioma are also called low grade glioma (LGG), and glioblastoma is also called glioblastoma multiforme (GBM). LGG patients frequently have isocitrate dehydrogenase (IDH) mutations. Patients with IDH mutations have been reported to have a better prognosis than patients without IDH mutations. Therefore, IDH mutations are an essential indicator for the classification of glioma. That is why we focused on the IDH1 mutation. In this paper, we aimed to classify the presence or absence of the IDH1 mutation using WSIs and clinical data of glioma patients. Ensemble learning between the WSIs model and the clinical data model is used to classify the presence or absence of IDH1 mutation. By using slide level labels, we combined patch-based imaging information from hematoxylin and eosin (H & E) stained WSIs, along with clinical data using deep image feature extraction and machine learning classifier for predicting IDH1 gene mutation prediction versus wild-type across cohort of 546 patients. We experimented with different deep learning (DL) models including attention-based multiple instance learning (ABMIL) models on imaging data along with gradient boosting machine (LightGBM) for the clinical variables. Further, we used hyperparameter optimization to find the best overall model in terms of classification accuracy. We obtained the highest area under the curve (AUC) of 0.823 for WSIs, 0.782 for clinical data, and 0.852 for ensemble results using MaxViT and LightGBM combination, respectively. Our experimental results indicate that the overall accuracy of the AI models can be improved by using both clinical data and images.

Reduction of NIFTI files storage and compression to facilitate telemedicine services based on quantization hiding of downsampling approach

Magnetic resonance imaging is a medical imaging technique to create comprehensive images of the tissues and organs in the body. This study presents an advanced approach for storing and compressing neuroimaging informatics technology initiative files, a standard format in magnetic resonance imaging. It is designed to enhance telemedicine services by facilitating efficient and high-quality communication between healthcare practitioners and patients. The proposed downsampling approach begins by opening the neuroimaging informatics technology initiative file as volumetric data and then planning it into several slice images. Then, the quantization hiding technique will be applied to each of the two consecutive slice images to generate the stego slice with the same size. This involves the following major steps: normalization, microblock generation, and discrete cosine transformation. Finally, it assembles the resultant stego slice images to produce the final neuroimaging informatics technology initiative file as volumetric data. The upsampling process, designed to be completely blind, reverses the downsampling steps to reconstruct the subsequent image slice accurately. The efficacy of the proposed method was evaluated using a magnetic resonance imaging dataset, focusing on peak signal-to-noise ratio, signal-to-noise ratio, structural similarity index, and Entropy as key performance metrics. The results demonstrate that the proposed approach not only significantly reduces file sizes but also maintains high image quality.

A multi-class brain tumor grading system based on histopathological images using a hybrid YOLO and RESNET networks

Gliomas are primary brain tumors caused by glial cells. These cancers' classification and grading are crucial for prognosis and treatment planning. Deep learning (DL) can potentially improve the digital pathology investigation of brain tumors. In this paper, we developed a technique for visualizing a predictive tumor grading model on histopathology pictures to help guide doctors by emphasizing characteristics and heterogeneity in forecasts. The proposed technique is a hybrid model based on YOLOv5 and ResNet50. The function of YOLOv5 is to localize and classify the tumor in large histopathological whole slide images (WSIs). The suggested technique incorporates ResNet into the feature extraction of the YOLOv5 framework, and the detection results show that our hybrid network is effective for identifying brain tumors from histopathological images. Next, we estimate the glioma grades using the extreme gradient boosting classifier. The high-dimensional characteristics and nonlinear interactions present in histopathology images are well-handled by this classifier. DL techniques have been used in previous computer-aided diagnosis systems for brain tumor diagnosis. However, by combining the YOLOv5 and ResNet50 architectures into a hybrid model specifically designed for accurate tumor localization and predictive grading within histopathological WSIs, our study presents a new approach that advances the field. By utilizing the advantages of both models, this creative integration goes beyond traditional techniques to produce improved tumor localization accuracy and thorough feature extraction. Additionally, our method ensures stable training dynamics and strong model performance by integrating ResNet50 into the YOLOv5 framework, addressing concerns about gradient explosion. The proposed technique is tested using the cancer genome atlas dataset. During the experiments, our model outperforms the other standard ways on the same dataset. Our results indicate that the proposed hybrid model substantially impacts tumor subtype discrimination between low-grade glioma (LGG) II and LGG III. With 97.2% of accuracy, 97.8% of precision, 98.6% of sensitivity, and the Dice similarity coefficient of 97%, the proposed model performs well in classifying four grades. These results outperform current approaches for identifying LGG from high-grade glioma and provide competitive performance in classifying four categories of glioma in the literature.

Advancements in artificial intelligence-driven techniques for interventional cardiology

This paper aims to thoroughly discuss the impact of artificial intelligence (AI) on clinical practice in interventional cardiology (IC) with special recognition of its most recent advancements. Thus, recent years have been exceptionally abundant in advancements in computational tools, including the development of AI. The application of AI development is currently in its early stages, nevertheless new technologies have proven to be a promising concept, particularly considering IC showing great impact on patient safety, risk stratification and outcomes during the whole therapeutic process. The primary goal is to achieve the integration of multiple cardiac imaging modalities, establish online decision support systems and platforms based on augmented and/or virtual realities, and finally to create automatic medical systems, providing electronic health data on patients. In a simplified way, two main areas of AI utilization in IC may be distinguished, namely, virtual and physical. Consequently, numerous studies have provided data regarding AI utilization in terms of automated interpretation and analysis from various cardiac modalities, including electrocardiogram, echocardiography, angiography, cardiac magnetic resonance imaging, and computed tomography as well as data collected during robotic-assisted percutaneous coronary intervention procedures. Thus, this paper aims to thoroughly discuss the impact of AI on clinical practice in IC with special recognition of its most recent advancements.

New research progress on 18F-FDG PET/CT radiomics for EGFR mutation prediction in lung adenocarcinoma: a review

Lung cancer, the most frequently diagnosed cancer worldwide, is the leading cause of cancer-associated deaths. In recent years, significant progress has been achieved in basic and clinical research concerning the epidermal growth factor receptor (EGFR), and the treatment of lung adenocarcinoma has also entered a new era of individualized, targeted therapies. However, the detection of lung adenocarcinoma is usually invasive. 18F-FDG PET/CT can be used as a noninvasive molecular imaging approach, and radiomics can acquire high-throughput data from standard images. These methods play an increasingly prominent role in diagnosing and treating cancers. Herein, we reviewed the progress in applying 18F-FDG PET/CT and radiomics in lung adenocarcinoma clinical research and how these data are analyzed via traditional statistics, machine learning, and deep learning to predict EGFR mutation status, all of which achieved satisfactory results. Traditional statistics extract features effectively, machine learning achieves higher accuracy with complex algorithms, and deep learning obtains significant results through end-to-end methods. Future research should combine these methods to achieve more accurate predictions, providing reliable evidence for the precision treatment of lung adenocarcinoma. At the same time, facing challenges such as data insufficiency and high algorithm complexity, future researchers must continuously explore and optimize to better apply to clinical practice.

A multi-center performance assessment for automated histopathological classification and grading of glioma using whole slide images

Accurate pathological classification and grading of gliomas is crucial in clinical diagnosis and treatment. The application of deep learning techniques holds promise for automated histological pathology diagnosis. In this study, we collected 733 whole slide images from four medical centers, of which 456 were used for model training, 150 for internal validation, and 127 for multi-center testing. The study includes 5 types of common gliomas. A subtask-guided multi-instance learning image-to-label training pipeline was employed. The pipeline leveraged "patch prompting" for the model to converge with reasonable computational cost. Experiments showed that an overall accuracy of 0.79 in the internal validation dataset. The performance on the multi-center testing dataset showed an overall accuracy to 0.73. The findings suggest a minor yet acceptable performance decrease in multi-center data, demonstrating the model's strong generalizability and establishing a robust foundation for future clinical applications.

Сell clusters isolation in glioblastomas and their functional and molecular characterization using new morphometric approaches

BACKGROUND

Digital pathology has come a long way in terms of creating tools to improve existing diagnostic approaches. However, several pathology fields, such as neuropathology, are still characterized by low coverage from machine learning tools and neural network analysis, which may be due to the complexity of the internal cellular and molecular structure of the corresponding neoplasms, including glioblastomas.

METHOD

In the framework of this study, using advanced proprietary tools for obtaining images of histological slides and their deep morphometric analysis, we studied samples of 198 patients with glioblastoma with the selection of morphometric cell clusters. Also, cells of each cluster were isolated, and their proliferative, migratory, invasive activity, survival ability, aerobic glycolysis activity, and chemo- and radioresistance were studied.

RESULTS

Four morphometric clusters were identified, including small-cell cluster, paracirculonuclear cluster, hypochromic cluster, and macronuclear cluster, which significantly differed in morphometric parameters and functional parameters. Hypochromic cluster cells demonstrated the highest proliferation activity; macronuclear cluster was the most active glucose consumer; paracirculonuclear cluster had the most prominent migratory and invasive activity and hypoxia resistance; small-cell cluster demonstrated predominantly average values of all parameters. Moreover, additional analysis revealed the presence of a separate subcluster of stem cell elements that correspond in their molecular properties to glioma stem cells and are present in all four clusters. It also turned out that several key molecular parameters of glioblastoma, such as mutational modifications in the EGFR, PDGFRA, and NF1 genes, along with the molecular GBM subtype, are significantly correlated with the identified cell clusters.

CONCLUSIONS

Thus, the results represent an up-and-coming innovation in the practical field of digital pathology and fundamental questions of glioma carcinogenesis.

Multiple therapeutic approaches of glioblastoma multiforme: From terminal to therapy

Glioblastoma multiforme (GBM) is an aggressive brain cancer showing poor prognosis. Currently, treatment methods of GBM are limited with adverse outcomes and low survival rate. Thus, advancements in the treatment of GBM are of utmost importance, which can be achieved in recent decades. However, despite aggressive initial treatment, most patients develop recurrent diseases, and the overall survival rate of patients is impossible to achieve. Currently, researchers across the globe target signaling events along with tumor microenvironment (TME) through different drug molecules to inhibit the progression of GBM, but clinically they failed to demonstrate much success. Herein, we discuss the therapeutic targets and signaling cascades along with the role of the organoids model in GBM research. Moreover, we systematically review the traditional and emerging therapeutic strategies in GBM. In addition, we discuss the implications of nanotechnologies, AI, and combinatorial approach to enhance GBM therapeutics.

Histopathological auxiliary system for brain tumour (HAS-Bt) based on weakly supervised learning using a WHO CNS5-style pipeline

PURPOSE

Classification and grading of central nervous system (CNS) tumours play a critical role in the clinic. When WHO CNS5 simplifies the histopathology diagnosis and places greater emphasis on molecular pathology, artificial intelligence (AI) has been widely used to meet the increased need for an automatic histopathology scheme that could liberate pathologists from laborious work. This study was to explore the diagnosis scope and practicality of AI.

METHODS

A one-stop Histopathology Auxiliary System for Brain tumours (HAS-Bt) is introduced based on a pipeline-structured multiple instance learning (pMIL) framework developed with 1,385,163 patches from 1038 hematoxylin and eosin (H&E) slides. The system provides a streamlined service including slide scanning, whole-slide image (WSI) analysis and information management. A logical algorithm is used when molecular profiles are available.

RESULTS

The pMIL achieved an accuracy of 0.94 in a 9-type classification task on an independent dataset composed of 268 H&E slides. Three auxiliary functions are developed and a built-in decision tree with multiple molecular markers is used to automatically formed integrated diagnosis. The processing efficiency was 443.0 s per slide.

CONCLUSION

HAS-Bt shows outstanding performance and provides a novel aid for the integrated neuropathological diagnostic workflow of brain tumours using CNS 5 pipeline.

The "Superoncogene" Myc at the Crossroad between Metabolism and Gene Expression in Glioblastoma Multiforme

The concept of the Myc (c-myc, n-myc, l-myc) oncogene as a canonical, DNA-bound transcription factor has consistently changed over the past few years. Indeed, Myc controls gene expression programs at multiple levels: directly binding chromatin and recruiting transcriptional coregulators; modulating the activity of RNA polymerases (RNAPs); and drawing chromatin topology. Therefore, it is evident that Myc deregulation in cancer is a dramatic event. Glioblastoma multiforme (GBM) is the most lethal, still incurable, brain cancer in adults, and it is characterized in most cases by Myc deregulation. Metabolic rewiring typically occurs in cancer cells, and GBM undergoes profound metabolic changes to supply increased energy demand. In nontransformed cells, Myc tightly controls metabolic pathways to maintain cellular homeostasis. Consistently, in Myc-overexpressing cancer cells, including GBM cells, these highly controlled metabolic routes are affected by enhanced Myc activity and show substantial alterations. On the other hand, deregulated cancer metabolism impacts Myc expression and function, placing Myc at the intersection between metabolic pathway activation and gene expression. In this review paper, we summarize the available information on GBM metabolism with a specific focus on the control of the Myc oncogene that, in turn, rules the activation of metabolic signals, ensuring GBM growth.

Near Lossless Compression for 3D Radiological Images Using Optimal Multilinear Singular Value Decomposition (3D-VOI-OMLSVD)

Storage and transmission of high-compression 3D radiological images that create high-quality reconstruction upon decompression are critical necessities for effective and efficient teleradiology. To cater to this need, we propose a near lossless 3D image volume compression method based on optimal multilinear singular value decomposition called "3D-VOI-OMLSVD." The proposed strategy first eliminates any blank 2D image slices from the 3D image volume and uses the selective bounding volume (SBV) to identify and extract the volume of Interest (VOI). Following this, the VOI is decomposed with an optimal multilinear singular value decomposition (OMLSVD) to obtain the corresponding core tensor, factor matrices, and singular values that are compressed with adaptive binary range coder (ABRC), integrated as an entropy encoder. The compressed file can be transferred or transmitted and then decompressed in order to reconstruct the original image. The resultant decompressed VOI is acquired by reversing the above process and then fusing it with the background, using the bound volume coordinates associated with the compressed 3D image. The proposed method performance was tested on a variety of 3D radiological images with different imaging modalities and dimensions using quantitative evaluation metrics such as the compression rate (CR), bit rate (BR), peak signal to noise ratio (PSNR), and structural similarity index (SSIM). Furthermore, we also investigate the impact of VOI extraction on the model performance, before comparing it with two popular compression methods, namely JPEG and JPEG2000. Our proposed method, 3D-VOI-OMLSVD, displayed a high CR value, with a maximum of 37.31, and a low BR, with the lowest reported to be 0.21. The SSIM score was consistently high, with an average performance of 0.9868, while using < 1 second for decoding the image. We observe that with VOI extraction, the compression rate increases manifold, and bit rate drops significantly, and thus reduces the encoding and decoding time to a great extent. Compared to JPEG and JPEG2000, our method consistently performs better in terms of higher CR and lower BR. The results indicate that the proposed compression methodology performs consistently to create high-quality image compressions, and overall gives a better outcome when compared against two state-of-the-art and widely used methods, JPEG and JPEG2000.

Artificial Intelligence and Precision Medicine: A New Frontier for the Treatment of Brain Tumors

Brain tumors are a widespread and serious neurological phenomenon that can be life- threatening. The computing field has allowed for the development of artificial intelligence (AI), which can mimic the neural network of the human brain. One use of this technology has been to help researchers capture hidden, high-dimensional images of brain tumors. These images can provide new insights into the nature of brain tumors and help to improve treatment options. AI and precision medicine (PM) are converging to revolutionize healthcare. AI has the potential to improve cancer imaging interpretation in several ways, including more accurate tumor genotyping, more precise delineation of tumor volume, and better prediction of clinical outcomes. AI-assisted brain surgery can be an effective and safe option for treating brain tumors. This review discusses various AI and PM techniques that can be used in brain tumor treatment. These new techniques for the treatment of brain tumors, i.e., genomic profiling, microRNA panels, quantitative imaging, and radiomics, hold great promise for the future. However, there are challenges that must be overcome for these technologies to reach their full potential and improve healthcare.

A hierarchical machine learning model based on Glioblastoma patients' clinical, biomedical, and image data to analyze their treatment plans

AIM OF STUDY

Glioblastoma Multiforme (GBM) is an aggressive brain cancer in adults that kills most patients in the first year due to ineffective treatment. Different clinical, biomedical, and image data features are needed to analyze GBM, increasing complexities. Besides, they lead to weak performances for machine learning models due to ignoring physicians' knowledge. Therefore, this paper proposes a hierarchical model based on Fuzzy C-mean (FCM) clustering, Wrapper feature selection, and twelve classifiers to analyze treatment plans.

METHODOLOGY/APPROACH

The proposed method finds the effectiveness of previous and current treatment plans, hierarchically determining the best decision for future treatment plans for GBM patients using clinical data, biomedical data, and different image data. A case study is presented based on the Cancer Genome Atlas Glioblastoma Multiforme dataset to prove the effectiveness of the proposed model. This dataset is analyzed using data preprocessing, experts' knowledge, and a feature reduction method based on the Principal Component Analysis. Then, the FCM clustering method is utilized to reinforce classifier learning.

OUTCOMES OF STUDY

The proposed model finds the best combination of Wrapper feature selection and classifier for each cluster based on different measures, including accuracy, sensitivity, specificity, precision, F-score, and G-mean according to a hierarchical structure. It has the best performance among other reinforced classifiers. Besides, this model is compatible with real-world medical processes for GBM patients based on clinical, biomedical, and image data.

Intraoperative Glioma Grading Using Neural Architecture Search and Multi-Modal Imaging

Glioma grading during surgery can help clinical treatment planning and prognosis, but intraoperative pathological examination of frozen sections is limited by the long processing time and complex procedures. Near-infrared fluorescence imaging provides chances for fast and accurate real-time diagnosis. Recently, deep learning techniques have been actively explored for medical image analysis and disease diagnosis. However, issues of near-infrared fluorescence images, including small-scale, noise, and low-resolution, increase the difficulty of training a satisfying network. Multi-modal imaging can provide complementary information to boost model performance, but simultaneously designing a proper network and utilizing the information of multi-modal data is challenging. In this work, we propose a novel neural architecture search method DLS-DARTS to automatically search for network architectures to handle these issues. DLS-DARTS has two learnable stems for multi-modal low-level feature fusion and uses a modified perturbation-based derivation strategy to improve the performance on the area under the curve and accuracy. White light imaging and fluorescence imaging in the first near-infrared window (650-900 nm) and the second near-infrared window (1,000-1,700 nm) are applied to provide multi-modal information on glioma tissues. In the experiments on 1,115 surgical glioma specimens, DLS-DARTS achieved an area under the curve of 0.843 and an accuracy of 0.634, which outperformed manually designed convolutional neural networks including ResNet, PyramidNet, and EfficientNet, and a state-of-the-art neural architecture search method for multi-modal medical image classification. Our study demonstrates that DLS-DARTS has the potential to help neurosurgeons during surgery, showing high prospects in medical image analysis.

A review: The detection of cancer cells in histopathology based on machine vision

Machine vision is being employed in defect detection, size measurement, pattern recognition, image fusion, target tracking and 3D reconstruction. Traditional cancer detection methods are dominated by manual detection, which wastes time and manpower, and heavily relies on the pathologists' skill and work experience. Therefore, these manual detection approaches are not convenient for the inheritance of domain knowledge, and are not suitable for the rapid development of medical care in the future. The emergence of machine vision can iteratively update and learn the domain knowledge of cancer cell pathology detection to achieve automated, high-precision, and consistent detection. Consequently, this paper reviews the use of machine vision to detect cancer cells in histopathology images, as well as the benefits and drawbacks of various detection approaches. First, we review the application of image preprocessing and image segmentation in histopathology for the detection of cancer cells, and compare the benefits and drawbacks of different algorithms. Secondly, for the characteristics of histopathological cancer cell images, the research progress of shape, color and texture features and other methods is mainly reviewed. Furthermore, for the classification methods of histopathological cancer cell images, the benefits and drawbacks of traditional machine vision approaches and deep learning methods are compared and analyzed. Finally, the above research is discussed and forecasted, with the expected future development tendency serving as a guide for future research.

A 3-miRNA Signature Enables Risk Stratification in Glioblastoma Multiforme Patients with Different Clinical Outcomes

Malignant gliomas constitute a complex disease phenotype that demands optimum decision-making as they are highly heterogeneous. Such inter-individual variability also renders optimum patient stratification extremely difficult. microRNA (hsa-miR-20a, hsa-miR-21, hsa-miR-21) expression levels were determined by RT-qPCR, upon FFPE tissue sample collection of glioblastoma multiforme patients (n = 37). In silico validation was then performed through discriminant analysis. Immunohistochemistry images from biopsy material were utilized by a hybrid deep learning system to further cross validate the distinctive capability of patient risk groups. Our standard-of-care treated patient cohort demonstrates no age- or sex- dependence. The expression values of the 3-miRNA signature between the low- (OS > 12 months) and high-risk (OS < 12 months) groups yield a p-value of <0.0001, enabling risk stratification. Risk stratification is validated by a. our random forest model that efficiently classifies (AUC = 97%) patients into two risk groups (low- vs. high-risk) by learning their 3-miRNA expression values, and b. our deep learning scheme, which recognizes those patterns that differentiate the images in question. Molecular-clinical correlations were drawn to classify low- (OS > 12 months) vs. high-risk (OS < 12 months) glioblastoma multiforme patients. Our 3-microRNA signature (hsa-miR-20a, hsa-miR-21, hsa-miR-10a) may further empower glioblastoma multiforme prognostic evaluation in clinical practice and enrich drug repurposing pipelines.

Artificial Intelligence in Brain Tumour Surgery-An Emerging Paradigm

Artificial intelligence (AI) platforms have the potential to cause a paradigm shift in brain tumour surgery. Brain tumour surgery augmented with AI can result in safer and more effective treatment. In this review article, we explore the current and future role of AI in patients undergoing brain tumour surgery, including aiding diagnosis, optimising the surgical plan, providing support during the operation, and better predicting the prognosis. Finally, we discuss barriers to the successful clinical implementation, the ethical concerns, and we provide our perspective on how the field could be advanced.

Ensemble based machine learning approach for prediction of glioma and multi-grade classification

Glioma is the most pernicious cancer of the nervous system, with histological grade influencing the survival of patients. Despite many studies on the multimodal treatment approach, survival time remains brief. In this study, a novel two-stage ensemble of an ensemble-type machine learning-based predictive framework for glioma detection and its histograde classification is proposed. In the proposed framework, five characteristics belonging to 135 subjects were considered: human telomerase reverse transcriptase (hTERT), chitinase-like protein (YKL-40), interleukin 6 (IL-6), tissue inhibitor of metalloproteinase-1 (TIMP-1) and neutrophil/lymphocyte ratio (NLR). These characteristics were examined using distinctive ensemble-based machine learning classifiers and combination strategies to develop a computer-aided diagnostic system for the non-invasive prediction of glioma cases and their grade. In the first stage, the analysis was conducted to classify glioma cases and control subjects. Machine learning approaches were applied in the second stage to classify the recognised glioma cases into three grades, from grade II, which has a good prognosis, to grade IV, which is also known as glioblastoma. All experiments were evaluated with a five-fold cross-validation method, and the classification results were analysed using different statistical parameters. The proposed approach obtained a high value of accuracy and other statistical parameters compared with other state-of-the-art machine learning classifiers. Therefore, the proposed framework can be utilised for designing other intervention strategies for the prediction of glioma cases and their grades.

Personalized brachytherapy dose reconstruction using deep learning

BACKGROUND AND PURPOSE

Accurate calculation of the absorbed dose delivered to the tumor and normal tissues improves treatment gain factor, which is the major advantage of brachytherapy over external radiation therapy. To address the simplifications of TG-43 assumptions that ignore the dosimetric impact of medium heterogeneities, we proposed a deep learning (DL)-based approach, which improves the accuracy while requiring a reasonable computation time.

MATERIALS AND METHODS

We developed a Monte Carlo (MC)-based personalized brachytherapy dosimetry simulator (PBrDoseSim), deployed to generate patient-specific dose distributions. A deep neural network (DNN) was trained to predict personalized dose distributions derived from MC simulations, serving as ground truth. The paired channel input used for the training is composed of dose distribution kernel in water medium along with the full-volumetric density maps obtained from CT images reflecting medium heterogeneity.

RESULTS

The predicted single-dwell dose kernels were in good agreement with MC-based kernels serving as reference, achieving a mean relative absolute error (MRAE) and mean absolute error (MAE) of 1.16 ± 0.42% and 4.2 ± 2.7 × 10^-4 (Gy.sec^-1/voxel), respectively. The MRAE of the dose volume histograms (DVHs) between the DNN and MC calculations in the clinical target volume were 1.8 ± 0.86%, 0.56 ± 0.56%, and 1.48 ± 0.72% for D90, V150, and V100, respectively. For bladder, sigmoid, and rectum, the MRAE of D5cc between the DNN and MC calculations were 2.7 ± 1.7%, 1.9 ± 1.3%, and 2.1 ± 1.7%, respectively.

CONCLUSION

The proposed DNN-based personalized brachytherapy dosimetry approach exhibited comparable performance to the MC method while overcoming the computational burden of MC calculations and oversimplifications of TG-43.

Application of deep learning in image recognition and diagnosis of gastric cancer

In recent years, artificial intelligence has been extensively applied in the diagnosis of gastric cancer based on medical imaging. In particular, using deep learning as one of the mainstream approaches in image processing has made remarkable progress. In this paper, we also provide a comprehensive literature survey using four electronic databases, PubMed, EMBASE, Web of Science, and Cochrane. The literature search is performed until November 2020. This article provides a summary of the existing algorithm of image recognition, reviews the available datasets used in gastric cancer diagnosis and the current trends in applications of deep learning theory in image recognition of gastric cancer. covers the theory of deep learning on endoscopic image recognition. We further evaluate the advantages and disadvantages of the current algorithms and summarize the characteristics of the existing image datasets, then combined with the latest progress in deep learning theory, and propose suggestions on the applications of optimization algorithms. Based on the existing research and application, the label, quantity, size, resolutions, and other aspects of the image dataset are also discussed. The future developments of this field are analyzed from two perspectives including algorithm optimization and data support, aiming to improve the diagnosis accuracy and reduce the risk of misdiagnosis.

Real-time intraoperative glioma diagnosis using fluorescence imaging and deep convolutional neural networks

Purpose

Surgery is the predominant treatment modality of human glioma but suffers difficulty on clearly identifying tumor boundaries in clinic. Conventional practice involves neurosurgeon’s visual evaluation and intraoperative histological examination of dissected tissues using frozen section, which is time-consuming and complex. The aim of this study was to develop fluorescent imaging coupled with artificial intelligence technique to quickly and accurately determine glioma in real-time during surgery.

Methods

Glioma patients (N = 23) were enrolled and injected with indocyanine green for fluorescence image–guided surgery. Tissue samples (N = 1874) were harvested from surgery of these patients, and the second near-infrared window (NIR-II, 1000–1700 nm) fluorescence images were obtained. Deep convolutional neural networks (CNNs) combined with NIR-II fluorescence imaging (named as FL-CNN) were explored to automatically provide pathological diagnosis of glioma in situ in real-time during patient surgery. The pathological examination results were used as the gold standard.

Results

The developed FL-CNN achieved the area under the curve (AUC) of 0.945. Comparing to neurosurgeons’ judgment, with the same level of specificity >80%, FL-CNN achieved a much higher sensitivity (93.8% versus 82.0%, P < 0.001) with zero time overhead. Further experiments demonstrated that FL-CNN corrected >70% of the errors made by neurosurgeons. FL-CNN was also able to rapidly predict grade and Ki-67 level (AUC 0.810 and 0.625) of tumor specimens intraoperatively.

Conclusion

Our study demonstrates that deep CNNs are better at capturing important information from fluorescence images than surgeons’ evaluation during patient surgery. FL-CNN is highly promising to provide pathological diagnosis intraoperatively and assist neurosurgeons to obtain maximum resection safely.

Trial registration

ChiCTR ChiCTR2000029402. Registered 29 January 2020, retrospectively registered

Supplementary Information

The online version contains supplementary material available at 10.1007/s00259-021-05326-y.

Are radiomics features universally applicable to different organs?

BACKGROUND

Many studies have successfully identified radiomics features reflecting macroscale tumor features and tumor microenvironment for various organs. There is an increased interest in applying these radiomics features found in a given organ to other organs. Here, we explored whether common radiomics features could be identified over target organs in vastly different environments.

METHODS

Four datasets of three organs were analyzed. One radiomics model was constructed from the training set (lungs, n = 401), and was further evaluated in three independent test sets spanning three organs (lungs, n = 59; kidneys, n = 48; and brains, n = 43). Intensity histograms derived from the whole organ were compared to establish organ-level differences. We constructed a radiomics score based on selected features using training lung data over the tumor region. A total of 143 features were computed for each tumor. We adopted a feature selection approach that favored stable features, which can also capture survival. The radiomics score was applied to three independent test data from lung, kidney, and brain tumors, and whether the score could be used to separate high- and low-risk groups, was evaluated.

RESULTS

Each organ showed a distinct pattern in the histogram and the derived parameters (mean and median) at the organ-level. The radiomics score trained from the lung data of the tumor region included seven features, and the score was only effective in stratifying survival for other lung data, not in other organs such as the kidney and brain. Eliminating the lung-specific feature (2.5 percentile) from the radiomics score led to similar results. There were no common features between training and test sets, but a common category of features (texture category) was identified.

CONCLUSION

Although the possibility of a generally applicable model cannot be excluded, we suggest that radiomics score models for survival were mostly specific for a given organ; applying them to other organs would require careful consideration of organ-specific properties.

Can artificial intelligence overtake human intelligence on the bumpy road towards glioma therapy?

Gliomas are one of the most devastating primary brain tumors which impose significant management challenges to the clinicians. The aggressive behaviour of gliomas is mainly attributed to their rapid proliferation, unravelled genomics and the blood-brain barrier which protects the tumor cells from chemotherapeutic regimens. Suspects of brain tumors are usually assessed by magnetic resonance imaging and computed tomography. These images allow surgeons to decide on the tumor grading, intra-operative pathology, feasibility of surgery, and treatment planning. All these data are compiled manually by physicians, wherein it takes time for the validation of results and concluding the treatment modality. In this context, the arrival of artificial intelligence in this era of personalized medicine, has proven promising performance in the diagnosis and management of gliomas. Starting from grading prediction till outcome evaluation, artificial intelligence-based forefronts have revolutionized oncological research. Interestingly, this approach has also been able to precisely differentiate tumor lesion from healthy tissues. However, till date, their utility in neuro-oncological field remains limited due to the issues pertaining to their reliability and transparency. Hence, to shed novel insights on the "clinical utility of this novel approach on glioma management" and to reveal "the black-boxes that have to be solved for fruitful application of artificial intelligence in neuro-oncology research", we provide in this review, a succinct description of the potential gear of artificial intelligence-based avenues in glioma treatment and the barriers that impede their rapid implementation in neuro-oncology.

Translating amyloid PET of different radiotracers by a deep generative model for interchangeability

It is challenging to compare amyloid PET images obtained with different radiotracers. Here, we introduce a new approach to improve the interchangeability of amyloid PET acquired with different radiotracers through image-level translation. Deep generative networks were developed using unpaired PET datasets, consisting of 203 [¹¹C]PIB and 850 [¹⁸F]florbetapir brain PET images. Using 15 paired PET datasets, the standardized uptake value ratio (SUVR) values obtained from pseudo-PIB or pseudo-florbetapir PET images translated using the generative networks was compared to those obtained from the original images. The generated amyloid PET images showed similar distribution patterns with original amyloid PET of different radiotracers. The SUVR obtained from the original [¹⁸F]florbetapir PET was lower than those obtained from the original [¹¹C]PIB PET. The translated amyloid PET images reduced the difference in SUVR. The SUVR obtained from the pseudo-PIB PET images generated from [¹⁸F]florbetapir PET showed a good agreement with those of the original PIB PET (ICC = 0.87 for global SUVR). The SUVR obtained from the pseudo-florbetapir PET also showed a good agreement with those of the original [¹⁸F]florbetapir PET (ICC = 0.85 for global SUVR). The ICC values between the original and generated PET images were higher than those between original [¹¹C]PIB and [¹⁸F]florbetapir images (ICC = 0.65 for global SUVR). Our approach provides the image-level translation of amyloid PET images obtained using different radiotracers. It may facilitate the clinical studies designed with variable amyloid PET images due to long-term clinical follow-up as well as multicenter trials by enabling the translation of different types of amyloid PET.

Automatic diagnosis for cysts and tumors of both jaws on panoramic radiographs using a deep convolution neural network

OBJECTIVES

The purpose of this study was to automatically diagnose odontogenic cysts and tumors of both jaws on panoramic radiographs using deep learning. We proposed a novel framework of deep convolution neural network (CNN) with data augmentation for detection and classification of the multiple diseases.

METHODS

We developed a deep CNN modified from YOLOv3 for detecting and classifying odontogenic cysts and tumors of both jaws. Our data set of 1282 panoramic radiographs comprised 350 dentigerous cysts (DCs), 302 periapical cysts (PCs), 300 odontogenic keratocysts (OKCs), 230 ameloblastomas (ABs), and 100 normal jaws with no disease. In addition, the number of radiographs was augmented 12-fold by flip, rotation, and intensity changes. We evaluated the classification performance of the developed CNN by calculating sensitivity, specificity, accuracy, and area under the curve (AUC) for diseases of both jaws.

RESULTS

The overall classification performance for the diseases improved from 78.2% sensitivity, 93.9% specificity,91.3% accuracy, and 0.86 AUC using the CNN with unaugmented data set to 88.9% sensitivity, 97.2% specificity, 95.6% accuracy, and 0.94 AUC using the CNN with augmented data set. CNN using augmented data set had the following sensitivities, specificities, accuracies, and AUCs: 91.4%, 99.2%, 97.8%, and 0.96 for DCs, 82.8%, 99.2%, 96.2%, and 0.92 for PCs, 98.4%,92.3%,94.0%, and 0.97 for OKCs, 71.7%, 100%, 94.3%, and 0.86 for ABs, and 100.0%, 95.1%, 96.0%, and 0.97 for normal jaws, respectively.

CONCLUSION

The CNN method we developed for automatically diagnosing odontogenic cysts and tumors of both jaws on panoramic radiographs using data augmentation showed high sensitivity, specificity, accuracy, and AUC despite the limited number of panoramic images involved.

The Application of Deep Convolutional Neural Networks to Brain Cancer Images: A Survey

In recent years, improved deep learning techniques have been applied to biomedical image processing for the classification and segmentation of different tumors based on magnetic resonance imaging (MRI) and histopathological imaging (H&E) clinical information. Deep Convolutional Neural Networks (DCNNs) architectures include tens to hundreds of processing layers that can extract multiple levels of features in image-based data, which would be otherwise very difficult and time-consuming to be recognized and extracted by experts for classification of tumors into different tumor types, as well as segmentation of tumor images. This article summarizes the latest studies of deep learning techniques applied to three different kinds of brain cancer medical images (histology, magnetic resonance, and computed tomography) and highlights current challenges in the field for the broader applicability of DCNN in personalized brain cancer care by focusing on two main applications of DCNNs: classification and segmentation of brain cancer tumors images.

Isocitrate dehydrogenase (IDH) status prediction in histopathology images of gliomas using deep learning

Mutations in isocitrate dehydrogenase genes IDH1 and IDH2 are frequently found in diffuse and anaplastic astrocytic and oligodendroglial tumours as well as in secondary glioblastomas. As IDH is a very important prognostic, diagnostic and therapeutic biomarker for glioma, it is of paramount importance to determine its mutational status. The haematoxylin and eosin (H&E) staining is a valuable tool in precision oncology as it guides histopathology-based diagnosis and proceeding patient's treatment. However, H&E staining alone does not determine the IDH mutational status of a tumour. Deep learning methods applied to MRI data have been demonstrated to be a useful tool in IDH status prediction, however the effectiveness of deep learning on H&E slides in the clinical setting has not been investigated so far. Furthermore, the performance of deep learning methods in medical imaging has been practically limited by small sample sizes currently available. Here we propose a data augmentation method based on the Generative Adversarial Networks (GAN) deep learning methodology, to improve the prediction performance of IDH mutational status using H&E slides. The H&E slides were acquired from 266 grade II-IV glioma patients from a mixture of public and private databases, including 130 IDH-wildtype and 136 IDH-mutant patients. A baseline deep learning model without data augmentation achieved an accuracy of 0.794 (AUC = 0.920). With GAN-based data augmentation, the accuracy of the IDH mutational status prediction was improved to 0.853 (AUC = 0.927) when the 3,000 GAN generated training samples were added to the original training set (24,000 samples). By integrating also patients' age into the model, the accuracy improved further to 0.882 (AUC = 0.931). Our findings show that deep learning methodology, enhanced by GAN data augmentation, can support physicians in gliomas' IDH status prediction.

Emerging MRI Techniques to Redefine Treatment Response in Patients With Glioblastoma

Glioblastoma is the most common and most malignant primary brain tumor. Despite aggressive multimodal treatment, its prognosis remains poor. Even with continuous developments in MRI, which has provided us with newer insights into the diagnosis and understanding of tumor biology, response assessment in the posttherapy setting remains challenging. We believe that the integration of additional information from advanced neuroimaging techniques can further improve the diagnostic accuracy of conventional MRI. In this article, we review the utility of advanced neuroimaging techniques such as diffusion-weighted imaging, diffusion tensor imaging, perfusion-weighted imaging, proton magnetic resonance spectroscopy, and chemical exchange saturation transfer in characterizing and evaluating treatment response in patients with glioblastoma. We will also discuss the existing challenges and limitations of using these techniques in clinical settings and possible solutions to avoiding pitfalls in study design, data acquisition, and analysis for future studies. LEVEL OF EVIDENCE: 2 TECHNICAL EFFICACY STAGE: 3 J. Magn. Reson. Imaging 2020;52:978-997.

Artificial Intelligence in the Management of Glioma: Era of Personalized Medicine

Purpose: Artificial intelligence (AI) has accelerated novel discoveries across multiple disciplines including medicine. Clinical medicine suffers from a lack of AI-based applications, potentially due to lack of awareness of AI methodology. Future collaboration between computer scientists and clinicians is critical to maximize the benefits of transformative technology in this field for patients. To illustrate, we describe AI-based advances in the diagnosis and management of gliomas, the most common primary central nervous system (CNS) malignancy.

Methods: Presented is a succinct description of foundational concepts of AI approaches and their relevance to clinical medicine, geared toward clinicians without computer science backgrounds. We also review novel AI approaches in the diagnosis and management of glioma.

Results: Novel AI approaches in gliomas have been developed to predict the grading and genomics from imaging, automate the diagnosis from histopathology, and provide insight into prognosis.

Conclusion: Novel AI approaches offer acceptable performance in gliomas. Further investigation is necessary to improve the methodology and determine the full clinical utility of these novel approaches.

Deep-dose: a voxel dose estimation method using deep convolutional neural network for personalized internal dosimetry

Personalized dosimetry with high accuracy is crucial owing to the growing interests in personalized medicine. The direct Monte Carlo simulation is considered as a state-of-art voxel-based dosimetry technique; however, it incurs an excessive computational cost and time. To overcome the limitations of the direct Monte Carlo approach, we propose using a deep convolutional neural network (CNN) for the voxel dose prediction. PET and CT image patches were used as inputs for the CNN with the given ground truth from direct Monte Carlo. The predicted voxel dose rate maps from the CNN were compared with the ground truth and dose rate maps generated voxel S-value (VSV) kernel convolution method, which is one of the common voxel-based dosimetry techniques. The CNN-based dose rate map agreed well with the ground truth with voxel dose rate errors of 2.54% ± 2.09%. The VSV kernel approach showed a voxel error of 9.97% ± 1.79%. In the whole-body dosimetry study, the average organ absorbed dose errors were 1.07%, 9.43%, and 34.22% for the CNN, VSV, and OLINDA/EXM dosimetry software, respectively. The proposed CNN-based dosimetry method showed improvements compared to the conventional dosimetry approaches and showed results comparable with that of the direct Monte Carlo simulation with significantly lower calculation time.