Patch-Based Segmentation with Spatial Consistency: Application to MS Lesions in Brain MRI

A Hybrid Intelligence Approach for Circulating Tumor Cell Enumeration in Digital Pathology by Using CNN and Weak Annotations

Counting the number of Circulating Tumor Cells (CTCs) for cancer screenings is currently done by cytopathologists with a heavy time and energy cost. AI, especially deep learning, has shown great potential in medical imaging domains. The aim of this paper is to develop a novel hybrid intelligence approach to automatically enumerate CTCs by combining cytopathologist expertise with the efficiency of deep learning convolutional neural networks (CNNs). This hybrid intelligence approach includes three major components: CNN based CTC detection/localization using weak annotations, CNN based CTC segmentation, and a classifier to ultimately determine CTCs. A support vector machine (SVM) was investigated for classification efficiency. The B-scale transform was also introduced to find the maximum sphericality of a given region. The SVM classifier was implemented to use a three-element vector as its input, including the B-scale (size), texture, and area values from the detection and segmentation results. We collected 466 fluoroscopic images for CTC detection/localization, 473 images for CTC segmentation and another 198 images with 323 CTCs as an independent data set for CTC enumeration. Precision and recall for CTC detection are 0.98 and 0.92, which is comparable with the state-of-the-art results that needed much larger and stricter training data sets. The counting error on an independent testing set was 2-3% and 9% (with/without B-scale) and performs much better than previous thresholding approaches with 30% of counting error rates. Recent publications prove facilitation of other types of research in object localization and segmentation are necessary.

How far MS lesion detection and segmentation are integrated into the clinical workflow? A systematic review

INTRODUCTION

Over the past few years, the deep learning community has developed and validated a plethora of tools for lesion detection and segmentation in Multiple Sclerosis (MS). However, there is an important gap between validating models technically and clinically. To this end, a six-step framework necessary for the development, validation, and integration of quantitative tools in the clinic was recently proposed under the name of the Quantitative Neuroradiology Initiative (QNI).

AIMS

Investigate to what extent automatic tools in MS fulfill the QNI framework necessary to integrate automated detection and segmentation into the clinical neuroradiology workflow.

METHODS

Adopting the systematic Cochrane literature review methodology, we screened and summarised published scientific articles that perform automatic MS lesions detection and segmentation. We categorised the retrieved studies based on their degree of fulfillment of QNI's six-steps, which include a tool's technical assessment, clinical validation, and integration.

RESULTS

We found 156 studies; 146/156 (94%) fullfilled the first QNI step, 155/156 (99%) the second, 8/156 (5%) the third, 3/156 (2%) the fourth, 5/156 (3%) the fifth and only one the sixth.

CONCLUSIONS

To date, little has been done to evaluate the clinical performance and the integration in the clinical workflow of available methods for MS lesion detection/segmentation. In addition, the socio-economic effects and the impact on patients' management of such tools remain almost unexplored.

Inpainting surgical occlusion from laparoscopic video sequences for robot-assisted interventions

Purpose

Medical technology for minimally invasive surgery has undergone a paradigm shift with the introduction of robot-assisted surgery. However, it is very difficult to track the position of the surgical tools in a surgical scene, so it is crucial to accurately detect and identify surgical tools. This task can be aided by deep learning-based semantic segmentation of surgical video frames. Furthermore, due to the limited working and viewing areas of these surgical instruments, there is a higher chance of complications from tissue injuries (e.g., tissue scars and tears).

Approach

With the aid of digital inpainting algorithms, we present an application that uses image segmentation to remove surgical instruments from laparoscopic/endoscopic video. We employ a modified U-Net architecture (U-NetPlus) to segment the surgical instruments. It consists of a redesigned decoder and a pre-trained VGG11 or VGG16 encoder. The decoder was modified by substituting an up-sampling operation based on nearest-neighbor interpolation for the transposed convolution operation. Furthermore, these interpolation weights do not need to be learned to perform upsampling, which eliminates the artifacts generated by the transposed convolution. In addition, we use a very fast and adaptable data augmentation technique to further enhance performance. The instrument segmentation mask is filled in (i.e., inpainted) by the tool removal algorithms using the previously acquired tool segmentation masks and either previous instrument-containing frames or instrument-free reference frames.

Results

We have shown the effectiveness of the proposed surgical tool segmentation/removal algorithms on a robotic instrument dataset from the MICCAI 2015 and 2017 EndoVis Challenge. We report a 90.20% DICE for binary segmentation, a 76.26% DICE for instrument part segmentation, and a 46.07% DICE for instrument type (i.e., all instruments) segmentation on the MICCAI 2017 challenge dataset using our U-NetPlus architecture, outperforming the results of earlier techniques used and tested on these data. In addition, we demonstrated the successful execution of the tool removal algorithm from surgical tool-free videos that contained moving surgical tools that were generated artificially.

Conclusions

Our application successfully separates and eliminates the surgical tool to reveal a view of the background tissue that was otherwise hidden by the tool, producing results that are visually similar to the actual data.

Using patching asymmetric regions to assess ischemic stroke lesion in neuro imaging

Ischemic stroke is a universal ailment that endangers the life of patients and makes them bedridden until death. Over a decade, doctors and radiologists have been dissecting patient status straightforwardly from the printouts of the slice images delivered by different diagnostic imaging modalities. Computed Tomography (CT) is a frequently used imaging strategy for therapeutic analysis and neuroanatomical investigations. The main objective of the paper is to develop a simple technique with less architectural complication and power consumption. The proposed work is to section the ischemic stroke lesion more efficiently from multi-succession CT images using patching the asymmetric region. The Hough transform segment and extracts the features from the asymmetric region of the CT image and finally, the random forest is implemented to classify the unusual tissues from the CT image dependent on their pathological properties. RF classifier has been trained for different parts of the cerebrum for fragmenting the stroke lesion. The acquired outcomes produce better segmentation accuracy when compared with different strategies. The overall efficiency of the proposed method determines the Ischemic stroke with an accuracy of 95% with an RF classifier. Hence this method can be used in the segmentation process of stroke lesions.

Simultaneous brain structure segmentation in magnetic resonance images using deep convolutional neural networks

In brain magnetic resonance imaging (MRI) examinations, rapidly acquired two-dimensional (2D) T1-weighted sagittal slices are typically used to confirm brainstem atrophy and the presence of signals in the posterior pituitary gland. Image segmentation is essential for the automatic evaluation of chronological changes in the brainstem and pituitary gland. Thus, the purpose of our study was to use deep learning to automatically segment internal organs (brainstem, corpus callosum, pituitary, cerebrum, and cerebellum) in midsagittal slices of 2D T1-weighted images. Deep learning for the automatic segmentation of seven regions in the images was accomplished using two different methods: patch-based segmentation and semantic segmentation. The networks used for patch-based segmentation were AlexNet, GoogLeNet, and ResNet50, whereas semantic segmentation was accomplished using SegNet, VGG16-weighted SegNet, and U-Net. The precision and Jaccard index were calculated, and the extraction accuracy of the six convolutional network (DCNN) systems was evaluated. The highest precision (0.974) was obtained with the VGG16-weighted SegNet, and the lowest precision (0.506) was obtained with ResNet50. Based on the data, calculation times, and Jaccard indices obtained in this study, segmentation on a 2D image may be considered a viable and effective approach. We found that the optimal automatic segmentation of organs (brainstem, corpus callosum, pituitary, cerebrum, and cerebellum) on brain sagittal T1-weighted images could be achieved using SegNet with VGG16.

Automatic brain lesion segmentation on standard magnetic resonance images: a scoping review

OBJECTIVES

Medical image analysis practices face challenges that can potentially be addressed with algorithm-based segmentation tools. In this study, we map the field of automatic MR brain lesion segmentation to understand the clinical applicability of prevalent methods and study designs, as well as challenges and limitations in the field.

DESIGN

Scoping review.

SETTING

Three databases (PubMed, IEEE Xplore and Scopus) were searched with tailored queries. Studies were included based on predefined criteria. Emerging themes during consecutive title, abstract, methods and whole-text screening were identified. The full-text analysis focused on materials, preprocessing, performance evaluation and comparison.

RESULTS

Out of 2990 unique articles identified through the search, 441 articles met the eligibility criteria, with an estimated growth rate of 10% per year. We present a general overview and trends in the field with regard to publication sources, segmentation principles used and types of lesions. Algorithms are predominantly evaluated by measuring the agreement of segmentation results with a trusted reference. Few articles describe measures of clinical validity.

CONCLUSIONS

The observed reporting practices leave room for improvement with a view to studying replication, method comparison and clinical applicability. To promote this improvement, we propose a list of recommendations for future studies in the field.

Pathomechanisms of HIV-Associated Cerebral Small Vessel Disease: A Comprehensive Clinical and Neuroimaging Protocol and Analysis Pipeline

Rationale: We provide an in-depth description of a comprehensive clinical, immunological, and neuroimaging study that includes a full image processing pipeline. This approach, although implemented in HIV infected individuals, can be used in the general population to assess cerebrovascular health.

Aims: In this longitudinal study, we seek to determine the effects of neuroinflammation due to HIV-1 infection on the pathomechanisms of cerebral small vessel disease (CSVD). The study focuses on the interaction of activated platelets, pro-inflammatory monocytes and endothelial cells and their impact on the neurovascular unit. The effects on the neurovascular unit are evaluated by a novel combination of imaging biomarkers.

Sample Size: We will enroll 110 HIV-infected individuals on stable combination anti-retroviral therapy for at least three months and an equal number of age-matched controls. We anticipate a drop-out rate of 20%.

Methods and Design: Subjects are followed for three years and evaluated by flow cytometric analysis of whole blood (to measure platelet activation, platelet monocyte complexes, and markers of monocyte activation), neuropsychological testing, and brain MRI at the baseline, 18- and 36-month time points. MRI imaging follows the recommended clinical small vessel imaging standards and adds several advanced sequences to obtain quantitative assessments of brain tissues including white matter microstructure, tissue susceptibility, and blood perfusion.

Discussion: The study provides further understanding of the underlying mechanisms of CSVD in chronic inflammatory disorders such as HIV infection. The longitudinal study design and comprehensive approach allows the investigation of quantitative changes in imaging metrics and their impact on cognitive performance.

A Practical Guide to Artificial Intelligence-Based Image Analysis in Radiology

The use of artificial intelligence (AI) is a powerful tool for image analysis that is increasingly being evaluated by radiology professionals. However, due to the fact that these methods have been developed for the analysis of nonmedical image data and data structure in radiology departments is not "AI ready", implementing AI in radiology is not straightforward. The purpose of this review is to guide the reader through the pipeline of an AI project for automated image analysis in radiology and thereby encourage its implementation in radiology departments. At the same time, this review aims to enable readers to critically appraise articles on AI-based software in radiology.

Neuro-fuzzy patch-wise R-CNN for multiple sclerosis segmentation

The segmentation of the lesion plays a core role in diagnosis and monitoring of multiple sclerosis (MS). Magnetic resonance imaging (MRI) is the most frequent image modality used to evaluate such lesions. Because of the massive amount of data, manual segmentation cannot be achieved within a sensible time that restricts the usage of accurate quantitative measurement in clinical practice. Therefore, the need for effective automated segmentation techniques is critical. However, a large spatial variability between the structure of brain lesions makes it more challenging. Recently, convolutional neural network (CNN), in particular, the region-based CNN (R-CNN), have attained tremendous progress within the field of object recognition because of its ability to learn and represent features. CNN has proven a last-breaking performance in various fields, such as object recognition, and has also gained more attention in brain imaging, especially in tissue and brain segmentation. In this paper, an automated technique for MS lesion segmentation is proposed, which is built on a 3D patch-wise R-CNN. The proposed system includes two stages: first, segmenting MS lesions in T2-w and FLAIR sequences using R-CNN, then an adaptive neuro-fuzzy inference system (ANFIS) is applied to fuse the results of the two modalities. To evaluate the performance of the proposed method, the public MICCAI2008 MS challenge dataset is employed to segment MS lesions. The experimental results show competitive results of the proposed method compared with the state-of-the-art MS lesion segmentation methods with an average total score of 83.25 and an average sensitivity of 61.8% on the MICCAI2008 testing set. Graphical Abstract The proposed system overview. First, the input of two modalities FLAIR and T2 are pre-processed to remove the skull and correct the bias field. Then 3D patches for lesion and non-lesion tissues are extracted and fed to R-CNN. Each R-CNN produces a probability map of the segmentation result that provides to ANFIS to fuse the results and obtain the final MS lesion segmentation. The MS lesions are shown on a pre-processed FLAIR image.

U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instruments from Laparoscopic Images

With the advent of robot-assisted surgery, there has been a paradigm shift in medical technology for minimally invasive surgery. However, it is very challenging to track the position of the surgical instruments in a surgical scene, and accurate detection & identification of surgical tools is paramount. Deep learning-based semantic segmentation in frames of surgery videos has the potential to facilitate this task. In this work, we modify the U-Net architecture by introducing a pre-trained encoder and re-design the decoder part, by replacing the transposed convolution operation with an upsampling operation based on nearest-neighbor (NN) interpolation. To further improve performance, we also employ a very fast and flexible data augmentation technique. We trained the framework on 8 × 225 frame sequences of robotic surgical videos available through the MICCAI 2017 EndoVis Challenge dataset and tested it on 8 × 75 frame and 2 × 300 frame videos. Using our U-NetPlus architecture, we report a 90.20% DICE for binary segmentation, 76.26% DICE for instrument part segmentation, and 46.07% for instrument type (i.e., all instruments) segmentation, outperforming the results of previous techniques implemented and tested on these data.

Voxel-Wise Logistic Regression and Leave-One-Source-Out Cross Validation for white matter hyperintensity segmentation

Many algorithms have been proposed for automated segmentation of white matter hyperintensities (WMH) in brain MRI. Yet, broad uptake of any particular algorithm has not been observed. In this work, we argue that this may be due to variable and suboptimal validation data and frameworks, precluding direct comparison of methods on heterogeneous data. As a solution, we present Leave-One-Source-Out Cross Validation (LOSO-CV), which leverages all available data for performance estimation, and show that this gives more realistic (lower) estimates of segmentation algorithm performance on data from different scanners. We also develop a FLAIR-only WMH segmentation algorithm: Voxel-Wise Logistic Regression (VLR), inspired by the open-source Lesion Prediction Algorithm (LPA). Our variant facilitates more accurate parameter estimation, and permits intuitive interpretation of model parameters. We illustrate the performance of the VLR algorithm using the LOSO-CV framework with a dataset comprising freely available data from several recent competitions (96 images from 7 scanners). The performance of the VLR algorithm (median Similarity Index of 0.69) is compared to its LPA predecessor (0.58), and the results of the VLR algorithm in the 2017 WMH Segmentation Competition are also presented.

Baby brain atlases

The baby brain is constantly changing due to its active neurodevelopment, and research into the baby brain is one of the frontiers in neuroscience. To help guide neuroscientists and clinicians in their investigation of this frontier, maps of the baby brain, which contain a priori knowledge about neurodevelopment and anatomy, are essential. "Brain atlas" in this review refers to a 3D-brain image with a set of reference labels, such as a parcellation map, as the anatomical reference that guides the mapping of the brain. Recent advancements in scanners, sequences, and motion control methodologies enable the creation of various types of high-resolution baby brain atlases. What is becoming clear is that one atlas is not sufficient to characterize the existing knowledge about the anatomical variations, disease-related anatomical alterations, and the variations in time-dependent changes. In this review, the types and roles of the human baby brain MRI atlases that are currently available are described and discussed, and future directions in the field of developmental neuroscience and its clinical applications are proposed. The potential use of disease-based atlases to characterize clinically relevant information, such as clinical labels, in addition to conventional anatomical labels, is also discussed.

An Advanced MRI Multi-Modalities Segmentation Methodology Dedicated to Multiple Sclerosis Lesions Exploration and Differentiation

Multiple sclerosis (MS) is one of the most common neurological diseases in young people. This paper dealt with an automatic biomedical aided tool involving volumetric segmentation of multiple sclerosis lesions. To meet this challenge, our proposed methodology requires one preliminary cerebral zones segmentation performed using a new Gaussian mixture model based on various databases atlases. Afterward, lesion segmentation begins with the estimation of a lesion map, which is then subjected to threshold constraints and refined by a new lesion expansion algorithm. The evaluation was carried out on four clinical databases integrating various clinical cases which had different lesion loads and were presented by a set of MRI modalities at several noise levels. The results compared with those of the existing methods proved excellent cerebral segmentation with dice averages close to 0.8 and sensitivity and specificity averages greater than 0.9. In addition, depending on the used database, the lesion segmentation recorded mean values were close to or greater than 0.8 for the different metrics. The detection error and outline error averages were about 0.3. Besides the ability to identify the lesions affecting the different parts of the brain, even those spreading in the gray matter, the proposed methodology identified the lesions cores and their surrounding vasogenic edema. This has been thoroughly tested and validated by highly qualified radiologists and neurologists. The evaluation of the resulting discriminations recorded values close to or greater than 0.9 for dice, sensitivity, and specificity. As a valuable benefit, a computer aided diagnosis tool could be offered to clinicians. It would help efficiently during the MS diagnosis and avoid several confusions. Besides, it could be used for longitudinal survey and henceforth extends to other pathologies that could be explored by MRI modalities, such as glioblastoma or alzheimer's disease.

Multiple Sclerosis Lesion Segmentation Using Joint Label Fusion

This paper adapts the joint label fusion (JLF) multi-atlas image segmentation algorithm to the problem of multiple sclerosis (MS) lesion segmentation in multi-modal MRI. Conventionally, JLF requires a set of atlas images to be co-registered to the target image using deformable registration. However, given the variable spatial distribution of lesions in the brain, whole-brain deformable registration is unlikely to line up lesions between atlases and the target image. As a solution, we propose to first pre-segment the target image using an intensity regression based technique, yielding a set of "candidate" lesions. Each "candidate" lesion is then matched to a set of similar lesions in the atlas based on location and size; and deformable registration and JLF are applied at the level of the "candidate" lesion. The approach is evaluated on a dataset of 74 subjects with MS and shown to improve Dice similarity coefficient with reference manual segmentation by 12% over intensity regression technique.

Validation of a Regression Technique for Segmentation of White Matter Hyperintensities in Alzheimer's Disease

Segmentation and volumetric quantification of white matter hyperintensities (WMHs) is essential in assessment and monitoring of the vascular burden in aging and Alzheimer's disease (AD), especially when considering their effect on cognition. Manually segmenting WMHs in large cohorts is technically unfeasible due to time and accuracy concerns. Automated tools that can detect WMHs robustly and with high accuracy are needed. Here, we present and validate a fully automatic technique for segmentation and volumetric quantification of WMHs in aging and AD. The proposed technique combines intensity and location features frommultiplemagnetic resonance imaging contrasts and manually labeled training data with a linear classifier to perform fast and robust segmentations. It provides both a continuous subject specific WMH map reflecting different levels of tissue damage and binary segmentations. Themethodwas used to detectWMHs in 80 elderly/AD brains (ADC data set) as well as 40 healthy subjects at risk of AD (PREVENT-AD data set). Robustness across different scanners was validated using ten subjects from ADNI2/GO study. Voxel-wise and volumetric agreements were evaluated using Dice similarity index (SI) and intra-class correlation (ICC), yielding ICC=0.96 , SI = 0.62±0.16 for ADC data set and ICC=0.78 , SI=0.51±0.15 for PREVENT-AD data set. The proposed method was robust in the independent sample yielding SI=0.64±0.17 with ICC=0.93 for ADNI2/GO subjects. The proposed method provides fast, accurate, and robust segmentations on previously unseen data from different models of scanners, making it ideal to study WMHs in large scale multi-site studies.

Longitudinal multiple sclerosis lesion segmentation: Resource and challenge

In conjunction with the ISBI 2015 conference, we organized a longitudinal lesion segmentation challenge providing training and test data to registered participants. The training data consisted of five subjects with a mean of 4.4 time-points, and test data of fourteen subjects with a mean of 4.4 time-points. All 82 data sets had the white matter lesions associated with multiple sclerosis delineated by two human expert raters. Eleven teams submitted results using state-of-the-art lesion segmentation algorithms to the challenge, with ten teams presenting their results at the conference. We present a quantitative evaluation comparing the consistency of the two raters as well as exploring the performance of the eleven submitted results in addition to three other lesion segmentation algorithms. The challenge presented three unique opportunities: (1) the sharing of a rich data set; (2) collaboration and comparison of the various avenues of research being pursued in the community; and (3) a review and refinement of the evaluation metrics currently in use. We report on the performance of the challenge participants, as well as the construction and evaluation of a consensus delineation. The image data and manual delineations will continue to be available for download, through an evaluation website2 as a resource for future researchers in the area. This data resource provides a platform to compare existing methods in a fair and consistent manner to each other and multiple manual raters.

A Semiautomatic Method for Multiple Sclerosis Lesion Segmentation on Dual-Echo MR Imaging: Application in a Multicenter Context

BACKGROUND AND PURPOSE

The automatic segmentation of MS lesions could reduce time required for image processing together with inter- and intraoperator variability for research and clinical trials. A multicenter validation of a proposed semiautomatic method for hyperintense MS lesion segmentation on dual-echo MR imaging is presented.

MATERIALS AND METHODS

The classification technique used is based on a region-growing approach starting from manual lesion identification by an expert observer with a final segmentation-refinement step. The method was validated in a cohort of 52 patients with relapsing-remitting MS, with dual-echo images acquired in 6 different European centers.

RESULTS

We found a mathematic expression that made the optimization of the method independent of the need for a training dataset. The automatic segmentation was in good agreement with the manual segmentation (dice similarity coefficient = 0.62 and root mean square error = 2 mL). Assessment of the segmentation errors showed no significant differences in algorithm performance between the different MR scanner manufacturers (P > .05).

CONCLUSIONS

The method proved to be robust, and no center-specific training of the algorithm was required, offering the possibility for application in a clinical setting. Adoption of the method should lead to improved reliability and less operator time required for image analysis in research and clinical trials in MS.