Regularized siamese neural network for unsupervised outlier detection on brain multiparametric magnetic resonance imaging: Application to epilepsy lesion screening

Artificial intelligence in epilepsy - applications and pathways to the clinic

Artificial intelligence (AI) is rapidly transforming health care, and its applications in epilepsy have increased exponentially over the past decade. Integration of AI into epilepsy management promises to revolutionize the diagnosis and treatment of this complex disorder. However, translation of AI into neurology clinical practice has not yet been successful, emphasizing the need to consider progress to date and assess challenges and limitations of AI. In this Review, we provide an overview of AI applications that have been developed in epilepsy using a variety of data modalities: neuroimaging, electroencephalography, electronic health records, medical devices and multimodal data integration. For each, we consider potential applications, including seizure detection and prediction, seizure lateralization, localization of the seizure-onset zone and assessment for surgical or neurostimulation interventions, and review the performance of AI tools developed to date. We also discuss methodological considerations and challenges that must be addressed to successfully integrate AI into clinical practice. Our goal is to provide an overview of the current state of the field and provide guidance for leveraging AI in future to improve management of epilepsy.

MP2RAGE vs. MPRAGE surface-based morphometry in focal epilepsy

In drug-resistant focal epilepsy, detecting epileptogenic lesions using MRI poses a critical diagnostic challenge. Here, we assessed the utility of MP2RAGE-a T1-weighted sequence with self-bias correcting properties commonly utilized in ultra-high field MRI-for the detection of epileptogenic lesions using a surface-based morphometry pipeline based on FreeSurfer, and compared it to the common approach using T1w MPRAGE, both at 3T. We included data from 32 patients with focal epilepsy (5 MRI-positive, 27 MRI-negative with lobar seizure onset hypotheses) and 94 healthy controls from two epilepsy centres. Surface-based morphological measures and intensities were extracted and evaluated in univariate GLM analyses as well as multivariate unsupervised 'novelty detection' machine learning procedures. The resulting prediction maps were analyzed over a range of possible thresholds using alternative free-response receiver operating characteristic (AFROC) methodology with respect to the concordance with predefined lesion labels or hypotheses on epileptogenic zone location. We found that MP2RAGE performs at least comparable to MPRAGE and that especially analysis of MP2RAGE image intensities may provide additional diagnostic information. Secondly, we demonstrate that unsupervised novelty-detection machine learning approaches may be useful for the detection of epileptogenic lesions (maximum AFROC AUC 0.58) when there is only a limited lesional training set available. Third, we propose a statistical method of assessing lesion localization performance in MRI-negative patients with lobar hypotheses of the epileptogenic zone based on simulation of a random guessing process as null hypothesis. Based on our findings, it appears worthwhile to study similar surface-based morphometry approaches in ultra-high field MRI (≥ 7 T).

Overcoming the Challenges in the Development and Implementation of Artificial Intelligence in Radiology: A Comprehensive Review of Solutions Beyond Supervised Learning

Artificial intelligence (AI) in radiology is a rapidly developing field with several prospective clinical studies demonstrating its benefits in clinical practice. In 2022, the Korean Society of Radiology held a forum to discuss the challenges and drawbacks in AI development and implementation. Various barriers hinder the successful application and widespread adoption of AI in radiology, such as limited annotated data, data privacy and security, data heterogeneity, imbalanced data, model interpretability, overfitting, and integration with clinical workflows. In this review, some of the various possible solutions to these challenges are presented and discussed; these include training with longitudinal and multimodal datasets, dense training with multitask learning and multimodal learning, self-supervised contrastive learning, various image modifications and syntheses using generative models, explainable AI, causal learning, federated learning with large data models, and digital twins.

Si-MSPDNet: A multiscale Siamese network with parallel partial decoders for the 3-D measurement of spines in 3D ultrasonic images

Early screening and frequent monitoring effectively decrease the risk of severe scoliosis, but radiation exposure is a consequence of traditional radiograph examinations. Additionally, traditional X-ray images on the coronal or sagittal plane have difficulty providing three-dimensional (3-D) information on spinal deformities. The Scolioscan system provides an innovative 3-D spine imaging approach via ultrasonic scanning, and its feasibility has been demonstrated in numerous studies. In this paper, to further examine the potential of spinal ultrasonic data for describing 3-D spinal deformities, we propose a novel deep-learning tracker named Si-MSPDNet for extracting widely employed landmarks (spinous process (SP)) in ultrasonic images of spines and establish a 3-D spinal profile to measure 3-D spinal deformities. Si-MSPDNet has a Siamese architecture. First, we employ two efficient two-stage encoders to extract features from the uncropped ultrasonic image and the patch centered on the SP cut from the image. Then, a fusion block is designed to strengthen the communication between encoded features and further refine them from channel and spatial perspectives. The SP is a very small target in ultrasonic images, so its representation is weak in the highest-level feature maps. To overcome this, we ignore the highest-level feature maps and introduce parallel partial decoders to localize the SP. The correlation evaluation in the traditional Siamese network is also expanded to multiple scales to enhance cooperation. Furthermore, we propose a binary guided mask based on vertebral anatomical prior knowledge, which can further improve the performance of our tracker by highlighting the potential region with SP. The binary-guided mask is also utilized for fully automatic initialization in tracking. We collected spinal ultrasonic data and corresponding radiographs on the coronal and sagittal planes from 150 patients to evaluate the tracking precision of Si-MSPDNet and the performance of the generated 3-D spinal profile. Experimental results revealed that our tracker achieved a tracking success rate of 100% and a mean IoU of 0.882, outperforming some commonly used tracking and real-time detection models. Furthermore, a high correlation existed on both the coronal and sagittal planes between our projected spinal curve and that extracted from the spinal annotation in X-ray images. The correlation between the tracking results of the SP and their ground truths on other projected planes was also satisfactory. More importantly, the difference in mean curvatures was slight on all projected planes between tracking results and ground truths. Thus, this study effectively demonstrates the promising potential of our 3-D spinal profile extraction method for the 3-D measurement of spinal deformities using 3-D ultrasound data.

On the evaluation of outlier detection and one-class classification: a comparative study of algorithms, model selection, and ensembles

It has been shown that unsupervised outlier detection methods can be adapted to the one-class classification problem (Janssens and Postma, in: Proceedings of the 18th annual Belgian-Dutch on machine learning, pp 56-64, 2009; Janssens et al. in: Proceedings of the 2009 ICMLA international conference on machine learning and applications, IEEE Computer Society, pp 147-153, 2009. 10.1109/ICMLA.2009.16). In this paper, we focus on the comparison of one-class classification algorithms with such adapted unsupervised outlier detection methods, improving on previous comparison studies in several important aspects. We study a number of one-class classification and unsupervised outlier detection methods in a rigorous experimental setup, comparing them on a large number of datasets with different characteristics, using different performance measures. In contrast to previous comparison studies, where the models (algorithms, parameters) are selected by using examples from both classes (outlier and inlier), here we also study and compare different approaches for model selection in the absence of examples from the outlier class, which is more realistic for practical applications since labeled outliers are rarely available. Our results showed that, overall, SVDD and GMM are top-performers, regardless of whether the ground truth is used for parameter selection or not. However, in specific application scenarios, other methods exhibited better performance. Combining one-class classifiers into ensembles showed better performance than individual methods in terms of accuracy, as long as the ensemble members are properly selected.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10618-023-00931-x.

Highly Performing Automatic Detection of Structural Chromosomal Abnormalities Using Siamese Architecture

The detection of structural chromosomal abnormalities (SCA) is crucial for diagnosis, prognosis and management of many genetic diseases and cancers. This detection, done by highly qualified medical experts, is tedious and time-consuming. We propose a highly performing and intelligent method to assist cytogeneticists to screen for SCA. Each chromosome is present in two copies that make up a pair of chromosomes. Usually, SCA are present in only one copy of the pair. Convolutional neural networks (CNN) with Siamese architecture are particularly relevant for evaluating similarities between two images, which is why we used this method to detect abnormalities between both chromosomes of a given pair. As a proof-of-concept, we first focused on a deletion occurring on chromosome 5 (del(5q)) observed in hematological malignancies. Using our dataset, we conducted several experiments without and with data augmentation on seven popular CNN models. Overall, performances obtained were very relevant for detecting deletions, particularly with Xception and InceptionResNetV2 models achieving 97.50% and 97.01% of F₁-score, respectively. We additionally demonstrated that these models successfully recognized another SCA, inversion inv(3), which is one of the most difficult SCA to detect. The performance improved when the training was applied on inversion inv(3) dataset, achieving 94.82% of F₁-score. The technique that we propose in this paper is the first highly performing method based on Siamese architecture that allows the detection of SCA. Our code is publicly available at: https://github.com/MEABECHAR/ChromosomeSiameseAD.

Application of Artificial Intelligence Methods for Imaging of Spinal Metastasis

Spinal metastasis is the most common malignant disease of the spine. Recently, major advances in machine learning and artificial intelligence technology have led to their increased use in oncological imaging. The purpose of this study is to review and summarise the present evidence for artificial intelligence applications in the detection, classification and management of spinal metastasis, along with their potential integration into clinical practice. A systematic, detailed search of the main electronic medical databases was undertaken in concordance with the PRISMA guidelines. A total of 30 articles were retrieved from the database and reviewed. Key findings of current AI applications were compiled and summarised. The main clinical applications of AI techniques include image processing, diagnosis, decision support, treatment assistance and prognostic outcomes. In the realm of spinal oncology, artificial intelligence technologies have achieved relatively good performance and hold immense potential to aid clinicians, including enhancing work efficiency and reducing adverse events. Further research is required to validate the clinical performance of the AI tools and facilitate their integration into routine clinical practice.

Machine learning applications on neuroimaging for diagnosis and prognosis of epilepsy: A review

Machine learning is playing an increasingly important role in medical image analysis, spawning new advances in the clinical application of neuroimaging. There have been some reviews on machine learning and epilepsy before, and they mainly focused on electrophysiological signals such as electroencephalography (EEG) and stereo electroencephalography (SEEG), while neglecting the potential of neuroimaging in epilepsy research. Neuroimaging has its important advantages in confirming the range of the epileptic region, which is essential in presurgical evaluation and assessment after surgery. However, it is difficult for EEG to locate the accurate epilepsy lesion region in the brain. In this review, we emphasize the interaction between neuroimaging and machine learning in the context of epilepsy diagnosis and prognosis. We start with an overview of epilepsy and typical neuroimaging modalities used in epilepsy clinics, MRI, DWI, fMRI, and PET. Then, we elaborate two approaches in applying machine learning methods to neuroimaging data: (i) the conventional machine learning approach combining manual feature engineering and classifiers, (ii) the deep learning approach, such as the convolutional neural networks and autoencoders. Subsequently, the application of machine learning on epilepsy neuroimaging, such as segmentation, localization, and lateralization tasks, as well as tasks directly related to diagnosis and prognosis are looked into in detail. Finally, we discuss the current achievements, challenges, and potential future directions in this field, hoping to pave the way for computer-aided diagnosis and prognosis of epilepsy.

A Quarter-split Domain-adaptive Network for EGFR Gene Mutation Prediction in Lung Cancer by Standardizing Heterogeneous CT image

Epidermal growth factor receptor (EGFR) gene mutation status is crucial for the treatment planning of lung cancer. The gold standard for detecting EGFR mutation status relies on invasive tumor biopsy and expensive gene sequencing. Recently, computed tomography (CT) images and deep learning have shown promising results in non-invasively predicting EGFR mutation in lung cancer. However, CT scanning parameters such as slice thickness vary largely between different scanners and centers, making the deep learning models very sensitive to noise and therefore not robust in clinical practice. In this study, we propose a novel QuarterNet_adaptive model to predict EGFR mutation in lung cancer, which is robust to CT images of different thicknesses. We propose two components: 1) a quarter-split network to sequentially learn local lung features from different lung lobes and global lung features; 2) a domain adaptive strategy to learn CT thickness-invariant features. Furthermore, we collected a large dataset including 1413 patients with both EGFR gene sequencing and CT images of various thicknesses to evaluate the performance of the proposed model. Finally, the QuarterNet_adaptive model achieved AUC over 0.88 regarding CT images of different thicknesses, which improves largely than state-of-the-art methods.Clinical relevance-We proposed a non-invasive model to detect EGFR gene mutation in lung cancer, which is robust to CT images of different thicknesses and can assist lung cancer treatment planning.

Unsupervised anomaly detection in MR images using multicontrast information

PURPOSE

Anomaly detection in magnetic resonance imaging (MRI) is to distinguish the relevant biomarkers of diseases from those of normal tissues. In this paper, an unsupervised algorithm is proposed for pixel-level anomaly detection in multicontrast MRI.

METHODS

A deep neural network is developed, which uses only normal MR images as training data. The network has the two stages of feature generation and density estimation. For feature generation, relevant features are extracted from multicontrast MR images by performing contrast translation and dimension reduction. For density estimation, the distributions of the extracted features are estimated by using Gaussian mixture model (GMM). The two processes are trained to estimate normative distributions well presenting large normal datasets. In test phases, the proposed method can detect anomalies by measuring log-likelihood that a test sample belongs to the estimated normative distributions.

RESULTS

The proposed method and its variants were applied to detect glioblastoma and ischemic stroke lesion. Comparison studies with six previous anomaly detection algorithms demonstrated that the proposed method achieved relevant improvements in quantitative and qualitative evaluations. Ablation studies by removing each module from the proposed framework validated the effectiveness of each proposed module.

CONCLUSION

The proposed deep learning framework is an effective tool to detect anomalies in multicontrast MRI. The unsupervised approaches would have great potentials in detecting various lesions where annotated lesion data collection is limited.

A Survey of Deep Network Techniques All Classifiers Can Adopt

Deep neural networks (DNNs) have introduced novel and useful tools to the machine learning community. Other types of classifiers can potentially make use of these tools as well to improve their performance and generality. This paper reviews the current state of the art for deep learning classifier technologies that are being used outside of deep neural networks. Non-neural network classifiers can employ many components found in DNN architectures. In this paper, we review the feature learning, optimization, and regularization methods that form a core of deep network technologies. We then survey non-neural network learning algorithms that make innovative use of these methods to improve classification performance. Because many opportunities and challenges still exist, we discuss directions that can be pursued to expand the area of deep learning for a variety of classification algorithms.

Clinical Application of Machine Learning Models for Brain Imaging in Epilepsy: A Review

Epilepsy is a common neurological disorder characterized by recurrent and disabling seizures. An increasing number of clinical and experimental applications of machine learning (ML) methods for epilepsy and other neurological and psychiatric disorders are available. ML methods have the potential to provide a reliable and optimal performance for clinical diagnoses, prediction, and personalized medicine by using mathematical algorithms and computational approaches. There are now several applications of ML for epilepsy, including neuroimaging analyses. For precise and reliable clinical applications in epilepsy and neuroimaging, the diverse ML methodologies should be examined and validated. We review the clinical applications of ML models for brain imaging in epilepsy obtained from a PubMed database search in February 2021. We first present an overview of typical neuroimaging modalities and ML models used in the epilepsy studies and then focus on the existing applications of ML models for brain imaging in epilepsy based on the following clinical aspects: (i) distinguishing individuals with epilepsy from healthy controls, (ii) lateralization of the temporal lobe epilepsy focus, (iii) the identification of epileptogenic foci, (iv) the prediction of clinical outcomes, and (v) brain-age prediction. We address the practical problems and challenges described in the literature and suggest some future research directions.

MADGAN: unsupervised medical anomaly detection GAN using multiple adjacent brain MRI slice reconstruction

BACKGROUND

Unsupervised learning can discover various unseen abnormalities, relying on large-scale unannotated medical images of healthy subjects. Towards this, unsupervised methods reconstruct a 2D/3D single medical image to detect outliers either in the learned feature space or from high reconstruction loss. However, without considering continuity between multiple adjacent slices, they cannot directly discriminate diseases composed of the accumulation of subtle anatomical anomalies, such as Alzheimer's disease (AD). Moreover, no study has shown how unsupervised anomaly detection is associated with either disease stages, various (i.e., more than two types of) diseases, or multi-sequence magnetic resonance imaging (MRI) scans.

RESULTS

We propose unsupervised medical anomaly detection generative adversarial network (MADGAN), a novel two-step method using GAN-based multiple adjacent brain MRI slice reconstruction to detect brain anomalies at different stages on multi-sequence structural MRI: (Reconstruction) Wasserstein loss with Gradient Penalty + 100 [Formula: see text] loss-trained on 3 healthy brain axial MRI slices to reconstruct the next 3 ones-reconstructs unseen healthy/abnormal scans; (Diagnosis) Average [Formula: see text] loss per scan discriminates them, comparing the ground truth/reconstructed slices. For training, we use two different datasets composed of 1133 healthy T1-weighted (T1) and 135 healthy contrast-enhanced T1 (T1c) brain MRI scans for detecting AD and brain metastases/various diseases, respectively. Our self-attention MADGAN can detect AD on T1 scans at a very early stage, mild cognitive impairment (MCI), with area under the curve (AUC) 0.727, and AD at a late stage with AUC 0.894, while detecting brain metastases on T1c scans with AUC 0.921.

CONCLUSIONS

Similar to physicians' way of performing a diagnosis, using massive healthy training data, our first multiple MRI slice reconstruction approach, MADGAN, can reliably predict the next 3 slices from the previous 3 ones only for unseen healthy images. As the first unsupervised various disease diagnosis, MADGAN can reliably detect the accumulation of subtle anatomical anomalies and hyper-intense enhancing lesions, such as (especially late-stage) AD and brain metastases on multi-sequence MRI scans.

An anomaly detection approach to identify chronic brain infarcts on MRI

The performance of current machine learning methods to detect heterogeneous pathology is limited by the quantity and quality of pathology in medical images. A possible solution is anomaly detection; an approach that can detect all abnormalities by learning how 'normal' tissue looks like. In this work, we propose an anomaly detection method using a neural network architecture for the detection of chronic brain infarcts on brain MR images. The neural network was trained to learn the visual appearance of normal appearing brains of 697 patients. We evaluated its performance on the detection of chronic brain infarcts in 225 patients, which were previously labeled. Our proposed method detected 374 chronic brain infarcts (68% of the total amount of brain infarcts) which represented 97.5% of the total infarct volume. Additionally, 26 new brain infarcts were identified that were originally missed by the radiologist during radiological reading. Our proposed method also detected white matter hyperintensities, anomalous calcifications, and imaging artefacts. This work shows that anomaly detection is a powerful approach for the detection of multiple brain abnormalities, and can potentially be used to improve the radiological workflow efficiency by guiding radiologists to brain anomalies which otherwise remain unnoticed.

Detection of covert lesions in focal epilepsy using computational analysis of multimodal magnetic resonance imaging data

Objective

To compare the location of suspect lesions detected by computational analysis of multimodal magnetic resonance imaging data with areas of seizure onset, early propagation, and interictal epileptiform discharges (IEDs) identified with stereoelectroencephalography (SEEG) in a cohort of patients with medically refractory focal epilepsy and radiologically normal magnetic resonance imaging (MRI) scans.

Methods

We developed a method of lesion detection using computational analysis of multimodal MRI data in a cohort of 62 control subjects, and 42 patients with focal epilepsy and MRI‐visible lesions. We then applied it to detect covert lesions in 27 focal epilepsy patients with radiologically normal MRI scans, comparing our findings with the areas of seizure onset, early propagation, and IEDs identified at SEEG.

Results

Seizure‐onset zones (SoZs) were identified at SEEG in 18 of the 27 patients (67%) with radiologically normal MRI scans. In 11 of these 18 cases (61%), concordant abnormalities were detected by our method. In the remaining seven cases, either early seizure propagation or IEDs were observed within the abnormalities detected, or there were additional areas of imaging abnormalities found by our method that were not sampled at SEEG. In one of the nine patients (11%) in whom SEEG was inconclusive, an abnormality, which may have been involved in seizures, was identified by our method and was not sampled at SEEG.

Significance

Computational analysis of multimodal MRI data revealed covert abnormalities in the majority of patients with refractory focal epilepsy and radiologically normal MRI that co‐located with SEEG defined zones of seizure onset. The method could help identify areas that should be targeted with SEEG when considering epilepsy surgery.

Unsupervised Tumor Characterization via Conditional Generative Adversarial Networks

Grading for cancer, based upon the degree of cancer differentiation, plays a major role in describing the characteristics and behavior of the cancer and determining treatment plan for patients. The grade is determined by a subjective and qualitative assessment of tissues under microscope, which suffers from high inter- and intra-observer variability among pathologists. Digital pathology offers an alternative means to automate the procedure as well as to improve the accuracy and robustness of cancer grading. However, most of such methods tend to mimic or reproduce cancer grade determined by human experts. Herein, we propose an alternative, quantitative means of assessing and characterizing cancers in an unsupervised manner. The proposed method utilizes conditional generative adversarial networks to characterize tissues. The proposed method is evaluated using whole slide images (WSIs) and tissue microarrays (TMAs) of colorectal cancer specimens. The results suggest that the proposed method holds a potential for quantifying cancer characteristics and improving cancer pathology.

Siamese Neural Networks for EEG-based Brain-computer Interfaces

Motivated by the inconceivable capability of human brain in simultaneously processing multi-modal signals and its real-time feedback to the outer world events, there has been a surge of interest in establishing a communication bridge between the human brain and a computer, which are referred to as Brain-computer Interfaces (BCI). To this aim, monitoring the electrical activity of brain through Electroencephalogram (EEG) has emerged as the prime choice for BCI systems. To discover the underlying and specific features of brain signals for different mental tasks, a considerable number of research works are developed based on statistical and data-driven techniques. However, a major bottleneck in development of practical and commercial BCI systems is their limited performance when the number of mental tasks for classification is increased. In this work, we propose a new EEG processing and feature extraction paradigm based on Siamese neural networks, which can be conveniently merged and scaled up for multi-class problems. The idea of Siamese networks is to train a double-input neural network based on a contrastive loss-function, which provides the capability of verifying if two input EEG trials are from the same class or not. In this work, a Siamese architecture, which is developed based on Convolutional Neural Networks (CNN) and provides a binary output on the similarity of two inputs, is combined with One vs. Rest (OVR) and One vs. One (OVO) techniques to scale up for multi-class problems. The efficacy of this architecture is evaluated on a 4-class Motor Imagery (MI) dataset from BCI Competition IV_2a and the results suggest a promising performance compared to its counterparts.

Investigations of Object Detection in Images/Videos Using Various Deep Learning Techniques and Embedded Platforms—A Comprehensive Review

In recent years there has been remarkable progress in one computer vision application area: object detection. One of the most challenging and fundamental problems in object detection is locating a specific object from the multiple objects present in a scene. Earlier traditional detection methods were used for detecting the objects with the introduction of convolutional neural networks. From 2012 onward, deep learning-based techniques were used for feature extraction, and that led to remarkable breakthroughs in this area. This paper shows a detailed survey on recent advancements and achievements in object detection using various deep learning techniques. Several topics have been included, such as Viola–Jones (VJ), histogram of oriented gradient (HOG), one-shot and two-shot detectors, benchmark datasets, evaluation metrics, speed-up techniques, and current state-of-art object detectors. Detailed discussions on some important applications in object detection areas, including pedestrian detection, crowd detection, and real-time object detection on Gpu-based embedded systems have been presented. At last, we conclude by identifying promising future directions.