A framework for evaluating image segmentation algorithms

Generation and evaluation of anatomy-preserving virtual CT for online adaptive proton therapy

BACKGROUND

Daily CTs generated by CBCT correction are required for daily replanning in online-adaptive proton therapy (APT) to effectively deal with inter-fractional changes. Out of the currently available methods, the suitability of a daily CT generation method for proton dose calculation also depends on the anatomical site.

PURPOSE

We propose an anatomy-preserving virtual CT (APvCT) method as a hybrid method of CBCT correction, which is especially suitable for large anatomy deformations. The accuracy of the hybrid method was assessed by comparison with the corrected CBCT (cCBCT) and virtual CT (vCT) methods in the context of online APT.

METHODS

Seventy-one daily CBCTs of four prostate cancer patients treated with intensity modulated proton therapy (IMPT) were converted to daily CTs using cCBCT, vCT, and the newly proposed APvCT method. In APvCT, planning CT (pCT) were mapped to CBCT geometry using deformable image registration with boundary conditions on controlling regions of interest (ROIs) created with deep learning segmentation on cCBCT. The relative frequency distribution (RFD) of HU, mass density and stopping power ratio (SPR) values were assessed and compared with the pCT. The ROIs in the APvCT and vCT were compared with cCBCT in terms of Dice similarity coefficient (DSC) and mean distance-to-agreement (mDTA). For each patient, a robustly optimized IMPT plan was created on the pCT and subsequent daily adaptive plans on daily CTs. For dose distribution comparison on the same anatomy, the daily adaptive plans on cCBCT and vCT were recalculated on the corresponding APvCT. The dose distributions were compared in terms of isodose volumes and 3D global gamma-index passing rate (GPR) at γ(2%, 2 mm) criterion.

RESULTS

For all patients, no noticeable difference in RFDs was observed amongst APvCT, vCT, and pCT except in cCBCT, which showed a noticeable difference. The minimum DSC value was 0.96 and 0.39 for contours in APvCT and vCT respectively. The average value of mDTA for APvCT was 0.01 cm for clinical target volume and ≤0.01 cm for organs at risk, which increased to 0.18 cm and ≤0.52 cm for vCT. The mean GPR value was 90.9%, 64.5%, and 67.0% for APvCT versus cCBCT, vCT versus cCBCT, and APvCT versus vCT, respectively. When recalculated on APvCT, the adaptive cCBCT and vCT plans resulted in mean GPRs of 89.5 ± 5.1% and 65.9 ± 19.1%, respectively. The mean DSC values for 80.0%, 90.0%, 95.0%, 98.0%, and 100.0% isodose volumes were 0.97, 0.97, 0.97, 0.95, and 0.91 for recalculated cCBCT plans, and 0.89, 0.88, 0.87, 0.85, and 0.81 for recalculated vCT plans. Hausdorff distance for the 100.0% isodose volume in some cases of recalculated cCBCT plans on APvCT exceeded 1.00 cm.

CONCLUSIONS

APvCT contours showed good agreement with reference contours of cCBCT which indicates anatomy preservation in APvCT. A vCT with erroneous anatomy can result in an incorrect adaptive plan. Further, slightly lower values of GPR between the APvCT and cCBCT-based adaptive plans can be explained by the difference in the cCBCT's SPR RFD from the pCT.

Multi-wavelength interference phase imaging for automatic breast cancer detection and delineation using diffuse reflection imaging

Millions of women globally are impacted by the major health problem of breast cancer (BC). Early detection of BC is critical for successful treatment and improved survival rates. In this study, we provide a progressive approach for BC detection using multi-wavelength interference (MWI) phase imaging based on diffuse reflection hyperspectral (HS) imaging. The proposed findings are based on the measurement of the interference pattern between the blue (446.6 nm) and red (632 nm) wavelengths. We consider implementing a comprehensive image processing and categorization method based on the use of Fast Fourier (FF) transform analysis pertaining to a change in the refractive index between tumor and normal tissue. We observed that cancer growth affects tissue organization dramatically, as seen by persistently increased refractive index variance in tumors compared normal areas. Both malignant and normal tissue had different depth data collected from it that was analyzed. To enhance the categorization of ex-vivo BC tissue, we developed and validated a training classifier algorithm specifically designed for categorizing HS cube data. Following the application of signal normalization with the FF transform algorithm, our methodology achieved a high level of performance with a specificity (Spec) of 94% and a sensitivity (Sen) of 90.9% for the 632 nm acquired image categorization, based on preliminary findings from breast specimens under investigation. Notably, we successfully leveraged unstained tissue samples to create 3D phase-resolved images that effectively highlight the distinctions in diffuse reflectance features between cancerous and healthy tissue. Preliminary data revealed that our imaging method might be able to assist specialists in safely excising malignant areas and assessing the tumor bed following resection automatically at different depths. This preliminary investigation might result in an effective "in-vivo" disease description utilizing optical technology using a typical RGB camera with wavelength-specific operation with our quantitative phase MWI imaging methodology.

Accuracy of TrUE-Net in comparison to established white matter hyperintensity segmentation methods: An independent validation study

White matter hyperintensities (WMH) are nearly ubiquitous in the aging brain, and their topography and overall burden are associated with cognitive decline. Given their numerosity, accurate methods to automatically segment WMH are needed. Recent developments, including the availability of challenge data sets and improved deep learning algorithms, have led to a new promising deep-learning based automated segmentation model called TrUE-Net, which has yet to undergo rigorous independent validation. Here, we compare TrUE-Net to six established automated WMH segmentation tools, including a semi-manual method. We evaluated the techniques at both global and regional level to compare their ability to detect the established relationship between WMH burden and age. We found that TrUE-Net was highly reliable at identifying WMH regions with low false positive rates, when compared to semi-manual segmentation as the reference standard. TrUE-Net performed similarly or favorably when compared to the other automated techniques. Moreover, TrUE-Net was able to detect relationships between WMH and age to a similar degree as the reference standard semi-manual segmentation at both the global and regional level. These results support the use of TrUE-Net for identifying WMH at the global or regional level, including in large, combined datasets.

GA-Net: A geographical attention neural network for the segmentation of body torso tissue composition

PURPOSE

Body composition analysis (BCA) of the body torso plays a vital role in the study of physical health and pathology and provides biomarkers that facilitate the diagnosis and treatment of many diseases, such as type 2 diabetes mellitus, cardiovascular disease, obstructive sleep apnea, and osteoarthritis. In this work, we propose a body composition tissue segmentation method that can automatically delineate those key tissues, including subcutaneous adipose tissue, skeleton, skeletal muscle tissue, and visceral adipose tissue, on positron emission tomography/computed tomography scans of the body torso.

METHODS

To provide appropriate and precise semantic and spatial information that is strongly related to body composition tissues for the deep neural network, first we introduce a new concept of the body area and integrate it into our proposed segmentation network called Geographical Attention Network (GA-Net). The body areas are defined following anatomical principles such that the whole body torso region is partitioned into three non-overlapping body areas. Each body composition tissue of interest is fully contained in exactly one specific minimal body area. Secondly, the proposed GA-Net has a novel dual-decoder schema that is composed of a tissue decoder and an area decoder. The tissue decoder segments the body composition tissues, while the area decoder segments the body areas as an auxiliary task. The features of body areas and body composition tissues are fused through a soft attention mechanism to gain geographical attention relevant to the body tissues. Thirdly, we propose a body composition tissue annotation approach that takes the body area labels as the region of interest, which significantly improves the reproducibility, precision, and efficiency of delineating body composition tissues.

RESULTS

Our evaluations on 50 low-dose unenhanced CT images indicate that GA-Net outperforms other architectures statistically significantly based on the Dice metric. GA-Net also shows improvements for the 95% Hausdorff Distance metric in most comparisons. Notably, GA-Net exhibits more sensitivity to subtle boundary information and produces more reliable and robust predictions for such structures, which are the most challenging parts to manually mend in practice, with potentially significant time-savings in the post hoc correction of these subtle boundary placement errors. Due to the prior knowledge provided from body areas, GA-Net achieves competitive performance with less training data. Our extension of the dual-decoder schema to TransUNet and 3D U-Net demonstrates that the new schema significantly improves the performance of these classical neural networks as well. Heatmaps obtained from attention gate layers further illustrate the geographical guidance function of body areas for identifying body tissues.

CONCLUSIONS

(i) Prior anatomic knowledge supplied in the form of appropriately designed anatomic container objects significantly improves the segmentation of bodily tissues. (ii) Of particular note are the improvements achieved in the delineation of subtle boundary features which otherwise would take much effort for manual correction. (iii) The method can be easily extended to existing networks to improve their accuracy for this application.

Hybrid UNet transformer architecture for ischemic stoke segmentation with MRI and CT datasets

A hybrid UNet and Transformer (HUT) network is introduced to combine the merits of the UNet and Transformer architectures, improving brain lesion segmentation from MRI and CT scans. The HUT overcomes the limitations of conventional approaches by utilizing two parallel stages: one based on UNet and the other on Transformers. The Transformer-based stage captures global dependencies and long-range correlations. It uses intermediate feature vectors from the UNet decoder and improves segmentation accuracy by enhancing the attention and relationship modeling between voxel patches derived from the 3D brain volumes. In addition, HUT incorporates self-supervised learning on the transformer network. This allows the transformer network to learn by maintaining consistency between the classification layers of the different resolutions of patches and augmentations. There is an improvement in the rate of convergence of the training and the overall capability of segmentation. Experimental results on benchmark datasets, including ATLAS and ISLES2018, demonstrate HUT's advantage over the state-of-the-art methods. HUT achieves higher Dice scores and reduced Hausdorff Distance scores in single-modality and multi-modality lesion segmentation. HUT outperforms the state-the-art network SPiN in the single-modality MRI segmentation on Anatomical Tracings of lesion After Stroke (ATLAS) dataset by 4.84% of Dice score and a large margin of 40.7% in the Hausdorff Distance score. HUT also performed well on CT perfusion brain scans in the Ischemic Stroke Lesion Segmentation (ISLES2018) dataset and demonstrated an improvement over the recent state-of-the-art network USSLNet by 3.3% in the Dice score and 12.5% in the Hausdorff Distance score. With the analysis of both single and multi-modalities datasets (ATLASR12 and ISLES2018), we show that HUT can perform and generalize well on different datasets. Code is available at: https://github.com/vicsohntu/HUT_CT.

Automated hippocampal segmentation algorithms evaluated in stroke patients

Deep learning segmentation algorithms can produce reproducible results in a matter of seconds. However, their application to more complex datasets is uncertain and may fail in the presence of severe structural abnormalities-such as those commonly seen in stroke patients. In this investigation, six recent, deep learning-based hippocampal segmentation algorithms were tested on 641 stroke patients of a multicentric, open-source dataset ATLAS 2.0. The comparisons of the volumes showed that the methods are not interchangeable with concordance correlation coefficients from 0.266 to 0.816. While the segmentation algorithms demonstrated an overall good performance (volumetric similarity [VS] 0.816 to 0.972, DICE score 0.786 to 0.921, and Hausdorff distance [HD] 2.69 to 6.34), no single out-performing algorithm was identified: FastSurfer performed best in VS, QuickNat in DICE and average HD, and Hippodeep in HD. Segmentation performance was significantly lower for ipsilesional segmentation, with a decrease in performance as a function of lesion size due to the pathology-based domain shift. Only QuickNat showed a more robust performance in volumetric similarity. Even though there are many pre-trained segmentation methods, it is important to be aware of the possible decrease in performance for the segmentation results on the lesion side due to the pathology-based domain shift. The segmentation algorithm should be selected based on the research question and the evaluation parameter needed. More research is needed to improve current hippocampal segmentation methods.

Glia Cell Morphology Analysis Using the Fiji GliaMorph Toolkit

Glial cells are the support cells of the nervous system. Glial cells typically have elaborate morphologies that facilitate close contacts with neighboring neurons, synapses, and the vasculature. In the retina, Müller glia (MG) are the principal glial cell type that supports neuronal function by providing a myriad of supportive functions via intricate cell morphologies and precise contacts. Thus, complex glial morphology is critical for glial function, but remains challenging to resolve at a sub-cellular level or reproducibly quantify in complex tissues. To address this issue, we developed GliaMorph as a Fiji-based macro toolkit that allows 3D glial cell morphology analysis in the developing and mature retina. As GliaMorph is implemented in a modular fashion, here we present guides to (a) setup of GliaMorph, (b) data understanding in 3D, including z-axis intensity decay and signal-to-noise ratio, (c) pre-processing data to enhance image quality, (d) performing and examining image segmentation, and (e) 3D quantification of MG features, including apicobasal texture analysis. To allow easier application, GliaMorph tools are supported with graphical user interfaces where appropriate, and example data are publicly available to facilitate adoption. Further, GliaMorph can be modified to meet users' morphological analysis needs for other glial or neuronal shapes. Finally, this article provides users with an in-depth understanding of data requirements and the workflow of GliaMorph. © 2023 The Authors. Current Protocols published by Wiley Periodicals LLC. Basic Protocol 1: Download and installation of GliaMorph components including example data Basic Protocol 2: Understanding data properties and quality 3D-essential for subsequent analysis and capturing data property issues early Basic Protocol 3: Pre-processing AiryScan microscopy data for analysis Alternate Protocol: Pre-processing confocal microscopy data for analysis Basic Protocol 4: Segmentation of glial cells Basic Protocol 5: 3D quantification of glial cell morphology.

MEED: A novel robust contrast enhancement procedure yielding highly-convergent thresholding of biofilm images

Morphological and statistical investigation of biofilm images may be even more critical than the image acquisition itself, in particular in the presence of morphologically complex distributions, due to the unavoidable impact of the measurement technique too. Hence, digital image pre-processing is mandatory for reliable feature extraction and enhancement preliminary to segmentation. Also, pattern recognition in automated deep learning (both supervised and unsupervised) models often requires a preliminary effective contrast-enhancement. However, no universal consensus exists on the optimal contrast enhancement approach. This paper presents and discusses a new general, robust, reproducible, accurate and easy to implement contrast enhancement procedure, briefly named MEED-procedure, able to work on images with different bacterial coverages and biofilm structures, coming from different imaging instrumentations (herein stereomicroscope and transmission microscope). It exploits a proper succession of basic morphological operations (erosion and dilation) and a horizontal line structuring element, to minimize the impact on size and shape of the even finer bacterial features. It systematically enhances the objects of interest, without histogram stretching and/or undesirable artifacts yielded by common automated methods. The quality of the MEED-procedure is ascertained by segmentation tests which demonstrate its robustness regarding the determination of threshold and convergence of the thresholding algorithm. Extensive validation tests over a rich image database, comparison with the literature and comprehensive discussion of the conceptual background support the superiority of the MEED-procedure over the existing methods and demonstrate it is not a routine application of morphological operators.

Time-warping for robust automated arterial wall-recognition and tracking from single-scan-line ultrasound signals

Current ultrasound methods for recognition and motion-tracking of arterial walls are suited for image-based B-mode or M-mode scans but not adequately robust for single-line image-free scans. We introduce a time-warping-based technique to address this need. Its performance was validated through simulations and in-vivo trials on 21 subjects. The method recognized wall locations with 100 % precision for simulated frames (SNR > 10 dB). Clustering detections for multiple frames achieved sensitivity >98 %, while it was ∼90 % without clustering. The absence of arterial walls was predicted with 100 % specificity. In-vivo results corroborated the performance outcomes yielding a sensitivity ≥94 %, precision ≥98 %, and specificity ≥98 % using the clustering scheme. Further, excellent frame-to-frame tracking accuracy (absolute error <3 %, RMSE <2 μm) was demonstrated. Image-free measurements of peak arterial distension agreed with the image-based ones, within an error of 1.08 ± 3.65 % and RMSE of 38 μm. The method discerned the presence of arterial walls in A-mode frames, robustly localized, and tracked them even when they were proximal to hyperechoic regions or slow-moving tissue structures. Unification of delineation techniques with the proposed methods facilitates a complete image-free framework for measuring arterial dynamics and the development of reliable A-mode devices.

Assistance System for the Teaching of Natural Numbers to Preschool Children with the Use of Artificial Intelligence Algorithms

This research was aimed at designing an image recognition system that can help increase children’s interest in learning natural numbers between 0 and 9. The research method used was qualitative descriptive, observing early childhood learning in a face-to-face education model, especially in the learning of numbers, with additional data from literature studies. For the development of the system, the cascade method was used, consisting of three stages: identification of the population, design of the artificial intelligence architecture, and implementation of the recognition system. The method of the system sought to replicate a mechanic that simulates a game, whereby the child trains the artificial intelligence algorithm such that it recognizes the numbers that the child draws on a blackboard. The system is expected to help increase the ability of children in their interest to learn numbers and identify the meaning of quantities to help improve teaching success with a fun and engaging teaching method for children. The implementation of learning in this system is expected to make it easier for children to learn to write, read, and conceive the quantities of numbers, in addition to exploring their potential, creativity, and interest in learning, with the use of technologies.

Recent Progress in Epicardial and Pericardial Adipose Tissue Segmentation and Quantification Based on Deep Learning: A Systematic Review

Epicardial and pericardial adipose tissues (EAT and PAT), which are located around the heart, have been linked to coronary atherosclerosis, cardiomyopathy, coronary artery disease, and other cardiovascular diseases. Additionally, the volume and thickness of EAT are good predictors of CVD risk levels. Manual quantification of these tissues is a tedious and error-prone process. This paper presents a comprehensive and critical overview of research on the epicardial and pericardial adipose tissue segmentation and quantification methods, evaluates their effectiveness in terms of segmentation time and accuracy, provides a critical comparison of the methods, and presents ongoing and future challenges in the field. Described methods are classified into pericardial adipose tissue segmentation, direct epicardial adipose tissue segmentation, and epicardial adipose tissue segmentation via pericardium delineation. A comprehensive categorization of the underlying methods is conducted with insights into their evolution from traditional image processing methods to recent deep learning-based methods. The paper also provides an overview of the research on the clinical significance of epicardial and pericardial adipose tissues as well as the terminology and definitions used in the medical literature.

Stratification by Tumor Grade Groups in a Holistic Evaluation of Machine Learning for Brain Tumor Segmentation

Accurate and consistent segmentation plays an important role in the diagnosis, treatment planning, and monitoring of both High Grade Glioma (HGG), including Glioblastoma Multiforme (GBM), and Low Grade Glioma (LGG). Accuracy of segmentation can be affected by the imaging presentation of glioma, which greatly varies between the two tumor grade groups. In recent years, researchers have used Machine Learning (ML) to segment tumor rapidly and consistently, as compared to manual segmentation. However, existing ML validation relies heavily on computing summary statistics and rarely tests the generalizability of an algorithm on clinically heterogeneous data. In this work, our goal is to investigate how to holistically evaluate the performance of ML algorithms on a brain tumor segmentation task. We address the need for rigorous evaluation of ML algorithms and present four axes of model evaluation-diagnostic performance, model confidence, robustness, and data quality. We perform a comprehensive evaluation of a glioma segmentation ML algorithm by stratifying data by specific tumor grade groups (GBM and LGG) and evaluate these algorithms on each of the four axes. The main takeaways of our work are-(1) ML algorithms need to be evaluated on out-of-distribution data to assess generalizability, reflective of tumor heterogeneity. (2) Segmentation metrics alone are limited to evaluate the errors made by ML algorithms and their describe their consequences. (3) Adoption of tools in other domains such as robustness (adversarial attacks) and model uncertainty (prediction intervals) lead to a more comprehensive performance evaluation. Such a holistic evaluation framework could shed light on an algorithm's clinical utility and help it evolve into a more clinically valuable tool.

A computer-aided method based on geometrical texture features for a precocious detection of fetal Hydrocephalus in ultrasound images

BACKGROUD

Hydrocephalus is the most common anomaly of the fetal head characterized by an excessive accumulation of fluid in the brain processing. The diagnostic process of fetal heads using traditional evaluation techniques are generally time consuming and error prone. Usually, fetal head size is computed using an ultrasound (US) image around 20-22 weeks, which is the gestational age (GA). Biometrical measurements are extracted and compared with ground truth charts to identify normal or abnormal growth.

METHODS

In this paper, an attempt has been made to enhance the Hydrocephalus characterization process by extracting other geometrical and textural features to design an efficient recognition system. The superiority of this work consists of the reduced time processing and the complexity of standard automatic approaches for routine examination. This proposed method requires practical insidiousness of the precocious discovery of fetuses' malformation to alert the experts about the existence of abnormal outcome. The first task is devoted to a proposed pre-processing model using a standard filtering and a segmentation scheme using a modified Hough transform (MHT) to detect the region of interest. Indeed, the obtained clinical parameters are presented to the principal component analysis (PCA) model in order to obtain a reduced number of measures which are employed in the classification stage.

RESULTS

Thanks to the combination of geometrical and statistical features, the classification process provided an important ability and an interesting performance achieving more than 96% of accuracy to detect pathological subjects in premature ages.

CONCLUSIONS

The experimental results illustrate the success and the accuracy of the proposed classification method for a factual diagnostic of fetal head malformation.

Integrating a learned probabilistic model with energy functional for ultrasound image segmentation

The segmentation of ultrasound (US) images is steadily growing in popularity, owing to the necessity of computer-aided diagnosis (CAD) systems and the advantages that this technique shows, such as safety and efficiency. The objective of this work is to separate the lesion from its background in US images. However, most US images contain poor quality, which is affected by the noise, ambiguous boundary, and heterogeneity. Moreover, the lesion region may be not salient amid the other normal tissues, which makes its segmentation a challenging problem. In this paper, an US image segmentation algorithm that combines the learned probabilistic model with energy functionals is proposed. Firstly, a learned probabilistic model based on the generalized linear model (GLM) reduces the false positives and increases the likelihood energy term of the lesion region. It yields a new probability projection that attracts the energy functional toward the desired region of interest. Then, boundary indicator and probability statistical-based energy functional are used to provide a reliable boundary for the lesion. Integrating probabilistic information into the energy functional framework can effectively overcome the impact of poor quality and further improve the accuracy of segmentation. To verify the performance of the proposed algorithm, 40 images are randomly selected in three databases for evaluation. The values of DICE coefficient, the Jaccard distance, root-mean-square error, and mean absolute error are 0.96, 0.91, 0.059, and 0.042, respectively. Besides, the initialization of the segmentation algorithm and the influence of noise are also analyzed. The experiment shows a significant improvement in performance. A. Description of the proposed paper. B. The main steps involved in the proposed method.

Quantifying lung and diaphragm morphology using radiographs in normative pediatric subjects, and predicting CT-derived lung volume

OBJECTIVE

To quantify the effect of age on two-dimensional (2D) radiographic lung and diaphragm morphology and determine if 2D radiographic lung measurements can be used to estimate computer tomography (CT)-derived lung volume in normative pediatric subjects.

MATERIALS AND METHODS

Digitally reconstructed radiographs (DRRs) were created using retrospective chest CT scans from 77 pediatric male and female subjects aged birth to 19 years. 2D lung and diaphragm measurements were made on the DRRs using custom MATLAB code, and Spearman correlations and exponential regression equations were used to relate 2D measurements with age. In addition, 3D lung volumes were segmented using CT scans, and power regression equations were fitted to predict each lung's CT-derived volume from 2D lung measurements. The coefficient of determination (R² ) and standard error of the estimate (SEE) were used to assess the precision of the predictive equations with p < .05 indicating statistical significance.

RESULTS

All 2D radiographic lung and diaphragm measurements showed statistically significant positive correlations with age (p < .01), including lung major axis (Spearman rho ≥  0.90). Precise estimations of CT-derived lung volumes can be made using 2D lung measurements (R²  ≥ 0.95), including lung major axis (R²  ≥ 0.97).

INTERPRETATIONS

The reported pediatric age-specific reference data on 2D lung and diaphragm morphology and growth rates could be clinically used to identify lung and diaphragm pathologies during chest X-ray evaluations. The simple, precise, and clinically adaptable radiographic method for estimating CT-derived lung volumes may be used when pulmonary function tests are not readily available or difficult to perform.

Evaluation of the Use of Single- and Multi-Magnification Convolutional Neural Networks for the Determination and Quantitation of Lesions in Nonclinical Pathology Studies

Digital pathology platforms with integrated artificial intelligence have the potential to increase the efficiency of the nonclinical pathologist's workflow through screening and prioritizing slides with lesions and highlighting areas with specific lesions for review. Herein, we describe the comparison of various single- and multi-magnification convolutional neural network (CNN) architectures to accelerate the detection of lesions in tissues. Different models were evaluated for defining performance characteristics and efficiency in accurately identifying lesions in 5 key rat organs (liver, kidney, heart, lung, and brain). Cohorts for liver and kidney were collected from TG-GATEs open-source repository, and heart, lung, and brain from internally selected R&D studies. Annotations were performed, and models were trained on each of the available lesion classes in the available organs. Various class-consolidation approaches were evaluated from generalized lesion detection to individual lesion detections. The relationship between the amount of annotated lesions and the precision/accuracy of model performance is elucidated. The utility of multi-magnification CNN implementations in specific tissue subtypes is also demonstrated. The use of these CNN-based models offers users the ability to apply generalized lesion detection to whole-slide images, with the potential to generate novel quantitative data that would not be possible with conventional image analysis techniques.

3D IFPN: Improved Feature Pyramid Network for Automatic Segmentation of Gastric Tumor

Automatic segmentation of gastric tumor not only provides image-guided clinical diagnosis but also assists radiologists to read images and improve the diagnostic accuracy. However, due to the inhomogeneous intensity distribution of gastric tumors in CT scans, the ambiguous/missing boundaries, and the highly variable shapes of gastric tumors, it is quite challenging to develop an automatic solution. This study designs a novel 3D improved feature pyramidal network (3D IFPN) to automatically segment gastric tumors in computed tomography (CT) images. To meet the challenges of this extremely difficult task, the proposed 3D IFPN makes full use of the complementary information within the low and high layers of deep convolutional neural networks, which is equipped with three types of feature enhancement modules: 3D adaptive spatial feature fusion (ASFF) module, single-level feature refinement (SLFR) module, and multi-level feature refinement (MLFR) module. The 3D ASFF module adaptively suppresses the feature inconsistency in different levels and hence obtains the multi-level features with high feature invariance. Then, the SLFR module combines the adaptive features and previous multi-level features at each level to generate the multi-level refined features by skip connection and attention mechanism. The MLFR module adaptively recalibrates the channel-wise and spatial-wise responses by adding the attention operation, which improves the prediction capability of the network. Furthermore, a stage-wise deep supervision (SDS) mechanism and a hybrid loss function are also embedded to enhance the feature learning ability of the network. CT volumes dataset collected in three Chinese medical centers was used to evaluate the segmentation performance of the proposed 3D IFPN model. Experimental results indicate that our method outperforms state-of-the-art segmentation networks in gastric tumor segmentation. Moreover, to explore the generalization for other segmentation tasks, we also extend the proposed network to liver tumor segmentation in CT images of the MICCAI 2017 Liver Tumor Segmentation Challenge.

Automatic detection and segmentation of lumbar vertebrae from X-ray images for compression fracture evaluation

For compression fracture detection and evaluation, an automatic X-ray image segmentation technique that combines deep-learning and level-set methods is proposed. Automatic segmentation is much more difficult for X-ray images than for CT or MRI images because they contain overlapping shadows of thoracoabdominal structures including lungs, bowel gases, and other bony structures such as ribs. Additional difficulties include unclear object boundaries, the complex shape of the vertebra, inter-patient variability, and variations in image contrast. Accordingly, a structured hierarchical segmentation method is presented that combines the advantages of two deep-learning methods. Pose-driven learning is used to selectively identify the five lumbar vertebrae in an accurate and robust manner. With knowledge of the vertebral positions, M-net is employed to segment the individual vertebra. Finally, fine-tuning segmentation is applied by combining the level-set method with the previously obtained segmentation results. The performance of the proposed method was validated by 160 lumbar X-ray images, resulting in a mean Dice similarity metric of 91.60±2.22%. The results show that the proposed method achieves accurate and robust identification of each lumbar vertebra and fine segmentation of individual vertebra.

Impact of FAPI-PET/CT on Target Volume Definition in Radiation Therapy of Locally Recurrent Pancreatic Cancer

Simple Summary

We demonstrate how manual target definition based on contrast-enhanced computed tomography is highly unreliable and inconsistent. In a second step, we used a novel positron emission tomography tracer, FAPI (68Ga-labeled fibroblast activation protein inhibitor) for target volume definition. FAPI-PET/CT contains biologic information as it visualizes cancer associated fibroblasts. The pioneering use of FAPI PET/CT in radiation treatment planning improved target definition in locally recurrent pancreatic cancer.

Abstract

(1) Background: A new radioactive positron emission tomography (PET) tracer uses inhibitors of fibroblast activation protein (FAPI) to visualize FAP-expressing cancer associated fibroblasts. Significant FAPI-uptake has recently been demonstrated in pancreatic cancer patients. Target volume delineation for radiation therapy still relies on often less precise conventional computed tomography (CT) imaging, especially in locally recurrent pancreatic cancer patients. The need for improvement in precise tumor detection and delineation led us to innovatively use the novel FAPI-PET/CT for radiation treatment planning. (2) Methods: Gross tumor volumes (GTVs) of seven locally recurrent pancreatic cancer cases were contoured by six radiation oncologists. In addition, FAPI-PET/CT was used to automatically delineate tumors. The interobserver variability in target definition was analyzed and FAPI-based automatic GTVs were compared to the manually defined GTVs. (3) Results: Target definition differed significantly between different radiation oncologists with mean dice similarity coefficients (DSCs) between 0.55 and 0.65. There was no significant difference between the volumes of automatic FAPI-GTVs based on the threshold of 2.0 and most of the manually contoured GTVs by radiation oncologists. (4) Conclusion: Due to its high tumor to background contrast, FAPI-PET/CT seems to be a superior imaging modality compared to the current gold standard contrast-enhanced CT in pancreatic cancer. For the first time, we demonstrate how FAPI-PET/CT could facilitate target definition and increases consistency in radiation oncology in pancreatic cancer.

Cranial organs at risk delineation: heterogenous practices in radiotherapy planning

BACKGROUND

Segmentation is a crucial step in treatment planning that directly impacts dose distribution and optimization. The aim of this study was to evaluate the inter-individual variability of common cranial organs at risk (OAR) delineation in neurooncology practice.

METHODS

Anonymized simulation contrast-enhanced CT and MR scans of one patient with a solitary brain metastasis was used for delineation and analysis. Expert professionals from 16 radiotherapy centers involved in brain structures delineation were asked to segment 9 OAR on their own treatment planning system. As reference, two experts in neurooncology, produced a unique consensual contour set according to guidelines. Overlap ratio, Kappa index (KI), volumetric ratio, Commonly Contoured Volume, Supplementary Contoured Volume were evaluated using Artiview™ v 2.8.2-according to occupation, seniority and level of expertise of all participants.

RESULTS

For the most frequently delineated and largest OAR, the mean KI are often good (0.8 for the parotid and the brainstem); however, for the smaller OAR, KI degrade (0.3 for the optic chiasm, 0.5% for the cochlea), with a significant discrimination (p < 0.01). The radiation oncologists, members of Association des Neuro-Oncologue d'Expression Française society performed better in all indicators compared to non-members (p < 0.01). Our exercise was effective in separating the different participating centers with 3 of the reported indicators (p < 0.01).

CONCLUSION

Our study illustrates the heterogeneity in normal structures contouring between professionals. We emphasize the need for cerebral OAR delineation harmonization-that is a major determinant of therapeutic ratio and clinical trials evaluation.

Automated ultrasound assessment of amniotic fluid index using deep learning

The estimation of antenatal amniotic fluid (AF) volume (AFV) is important as it offers crucial information about fetal development, fetal well-being, and perinatal prognosis. However, AFV measurement is cumbersome and patient specific. Moreover, it is heavily sonographer-dependent, with measurement accuracy varying greatly depending on the sonographer's experience. Therefore, the development of accurate, robust, and adoptable methods to evaluate AFV is highly desirable. In this regard, automation is expected to reduce user-based variability and workload of sonographers. However, automating AFV measurement is very challenging, because accurate detection of AF pockets is difficult owing to various confusing factors, such as reverberation artifact, AF mimicking region and floating matter. Furthermore, AF pocket exhibits an unspecified variety of shapes and sizes, and ultrasound images often show missing or incomplete structural boundaries. To overcome the abovementioned difficulties, we develop a hierarchical deep-learning-based method, which consider clinicians' anatomical-knowledge-based approaches. The key step is the segmentation of the AF pocket using our proposed deep learning network, AF-net. AF-net is a variation of U-net combined with three complementary concepts - atrous convolution, multi-scale side-input layer, and side-output layer. The experimental results demonstrate that the proposed method provides a measurement of the amniotic fluid index (AFI) that is as robust and precise as the results from clinicians. The proposed method achieved a Dice similarity of 0.877±0.086 for AF segmentation and achieved a mean absolute error of 2.666±2.986 and mean relative error of 0.018±0.023 for AFI value. To the best of our knowledge, our method, for the first time, provides an automated measurement of AFI.

Volumetric and dosimetric impact of post-surgical MRI-guided radiotherapy for glioblastoma: A pilot study

Objectives:

Glioblastoma (GBM) radiotherapy (RT) target delineation requires MRI, ideally concurrent with CT simulation (pre-RT MRI). Due to limited MRI availability, <72 h post-surgery MRI is commonly used instead. Whilst previous investigations assessed volumetric differences between post-surgical and pre-RT delineations, dosimetric impact remains unknown. We quantify volumetric and dosimetric impact of using post-surgical MRI for GBM target delineation.

Methods:

Gross tumour volumes (GTVs) for five GBM patients receiving chemo-RT with post-surgical and pre-RT MRIs were delineated by three independent observers. Planning target volumes (PTVs) and RT plans were generated for each GTV. Volumetric and dosimetric differences were assessed through: absolute volumes, volume-distance histograms and dose-volume histogram statistics.

Results:

Post-surgical MRI delineations had significantly (p < 0.05) larger GTV and PTV volumes (median 16.7 and 64.4 cm^3, respectively). Post-surgical RT plans, applied to pre-RT delineations, had significantly decreased (p < 0.01) median PTV doses (ΔD99% = −8.1 Gy and ΔD95% = −2.0 Gy). Median organ-at-risk (OAR) dose increases (brainstem ΔD5% =+0.8, normal brain mean dose =+2.9 and normal brain ΔD10% = 5.3 Gy) were observed.

Conclusion:

Post-surgical MRI delineation significantly impacted RT planning, with larger normal-appearing tissue volumes irradiated and increased OAR doses, despite a reduced coverage of the pre-RT defined target.

Advances in knowledge:

We believe this is the first investigation assessing the dosimetric impact of using post-surgical MRI for GBM target delineation. It highlights the potential of significantly degraded RT plans, showing the clinical need for dedicated MRI for GBM RT.

A Validation Study of Liver Volumetry Estimation by a Semiautomated Software in Patients Undergoing Hepatic Resections

Purpose The purpose of this study was to validate the use of a semiautomated software for liver volumetry preoperatively by comparing it with the volume of resected specimen in patients undergoing hepatic resections.

Materials and Methods This is a single-center retrospective study of patients who underwent estimation of future liver remnant (FLR) using Myrian XP-Liver which is a semiautomated software for hepatectomy. The estimated resection volume, which is the sum of volume of normal liver to be resected and tumor volume, was compared with actual specimen weight to calculate the accuracy of the software. The statistical analysis was performed with SPSS software version 24.

Results Data on FLR estimation using the semiautomated software was available for 200 out of 388 patients who underwent formal hepatic resections. The median resected volume of surgical specimen was 650 mL (interquartile range [IQR] 364–950), while the median estimated volume using the Myrian software was 617 mL (IQR 362–979). There was significant correlation between estimated resection volume calculated using the semiautomated method and actual specimen weight (p-value < 0.0001) with the Spearman’s correlation value of 0.956.

Conclusion The estimated volume of liver to be resected as calculated by the semiautomated software was accurate and correlated significantly with the volume of resected specimen, and hence, the estimation of FLR volume may likely correlate with the true postoperative residual liver volume. In addition, the software-based liver segmentation, FLR estimation, and color-coded three-dimensional images provide a clear road map to the surgeon to facilitate safe resection.

Review of Deep Learning Approaches for the Segmentation of Multiple Sclerosis Lesions on Brain MRI

In recent years, there have been multiple works of literature reviewing methods for automatically segmenting multiple sclerosis (MS) lesions. However, there is no literature systematically and individually review deep learning-based MS lesion segmentation methods. Although the previous review also included methods based on deep learning, there are some methods based on deep learning that they did not review. In addition, their review of deep learning methods did not go deep into the specific categories of Convolutional Neural Network (CNN). They only reviewed these methods in a generalized form, such as supervision strategy, input data handling strategy, etc. This paper presents a systematic review of the literature in automated multiple sclerosis lesion segmentation based on deep learning. Algorithms based on deep learning reviewed are classified into two categories through their CNN style, and their strengths and weaknesses will also be given through our investigation and analysis. We give a quantitative comparison of the methods reviewed through two metrics: Dice Similarity Coefficient (DSC) and Positive Predictive Value (PPV). Finally, the future direction of the application of deep learning in MS lesion segmentation will be discussed.

Variability and reproducibility in deep learning for medical image segmentation

Medical image segmentation is an important tool for current clinical applications. It is the backbone of numerous clinical diagnosis methods, oncological treatments and computer-integrated surgeries. A new class of machine learning algorithm, deep learning algorithms, outperforms the results of classical segmentation in terms of accuracy. However, these techniques are complex and can have a high range of variability, calling the reproducibility of the results into question. In this article, through a literature review, we propose an original overview of the sources of variability to better understand the challenges and issues of reproducibility related to deep learning for medical image segmentation. Finally, we propose 3 main recommendations to address these potential issues: (1) an adequate description of the framework of deep learning, (2) a suitable analysis of the different sources of variability in the framework of deep learning, and (3) an efficient system for evaluating the segmentation results.

TUM-ParticleTyper: A detection and quantification tool for automated analysis of (Microplastic) particles and fibers

TUM-ParticleTyper is a novel program for the automated detection, quantification and morphological characterization of fragments, including particles and fibers, in images from optical, fluorescence and electron microscopy (SEM). It can be used to automatically select targets for subsequent chemical analysis, e.g., Raman microscopy, or any other single particle identification method. The program was specifically developed and validated for the analysis of microplastic particles on gold coated polycarbonate filters. Our method development was supported by the design of a filter holder that minimizes filter roughness and facilitates enhanced focusing for better images and Raman measurements. The TUM-ParticleTyper software is tunable to the user's specific sample demands and can extract the morphological characteristics of detected objects (coordinates, Feret's diameter min / max, area and shape). Results are saved in csv-format and contours of detected objects are displayed as an overlay on the original image. Additionally, the program can stitch a set of images to create a full image out of several smaller ones. An additional useful feature is the inclusion of a statistical process to calculate the minimum number of particles that must be chemically identified to be representative of all particles localized on the substrate. The program performance was evaluated on genuine microplastic samples. The TUM-ParticleTyper software localizes particles using an adaptive threshold with results comparable to the "gold standard" method (manual localization by an expert) and surpasses the commonly used Otsu thresholding by doubling the rate of true positive localizations. This enables the analysis of a statistically significant number of particles on the filter selected by random sampling, measured via single point approach. This extreme reduction in measurement points was validated by comparison to chemical imaging, applying both procedures to the same area at comparable processing times. The single point approach was both faster and more accurate proving the applicability of the presented program.

Deep Convolutional Neural Networks-Based Automatic Breast Segmentation and Mass Detection in DCE-MRI

Breast segmentation and mass detection in medical images are important for diagnosis and treatment follow-up. Automation of these challenging tasks can assist radiologists by reducing the high manual workload of breast cancer analysis. In this paper, deep convolutional neural networks (DCNN) were employed for breast segmentation and mass detection in dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI). First, the region of the breasts was segmented from the remaining body parts by building a fully convolutional neural network based on U-Net++. Using the method of deep learning to extract the target area can help to reduce the interference external to the breast. Second, a faster region with convolutional neural network (Faster RCNN) was used for mass detection on segmented breast images. The dataset of DCE-MRI used in this study was obtained from 75 patients, and a 5-fold cross validation method was adopted. The statistical analysis of breast region segmentation was carried out by computing the Dice similarity coefficient (DSC), Jaccard coefficient, and segmentation sensitivity. For validation of breast mass detection, the sensitivity with the number of false positives per case was computed and analyzed. The Dice and Jaccard coefficients and the segmentation sensitivity value for breast region segmentation were 0.951, 0.908, and 0.948, respectively, which were better than those of the original U-Net algorithm, and the average sensitivity for mass detection achieved 0.874 with 3.4 false positives per case.

Development of a new fully three-dimensional methodology for tumours delineation in functional images

Delineation of tumours in Positron Emission Tomography (PET) plays a crucial role in accurate diagnosis and radiotherapy treatment planning. In this context, it is of outmost importance to devise efficient and operator-independent segmentation algorithms capable of reconstructing the tumour three-dimensional (3D) shape. In previous work, we proposed a system for 3D tumour delineation on PET data (expressed in terms of Standardized Uptake Value - SUV), based on a two-step approach. Step 1 identified the slice enclosing the maximum SUV and generated a rough contour surrounding it. Such contour was then used to initialize step 2, where the 3D shape of the tumour was obtained by separately segmenting 2D PET slices, leveraging the slice-by-slice marching approach. Additionally, we combined active contours and machine learning components to improve performance. Despite its success, the slice marching approach poses unnecessary limitations that are naturally removed by performing the segmentation directly in 3D. In this paper, we migrate our system into 3D. In particular, the segmentation in step 2 is now performed by evolving an active surface directly in the 3D space. The key points of such an advancement are that it performs the shape reconstruction on the whole stack of slices simultaneously, naturally leveraging cross-slice information that could not be exploited before. Additionally, it does not require any specific stopping condition, as the active surface naturally reaches a stable topology once convergence is achieved. Performance of this fully 3D approach is evaluated on the same dataset discussed in our previous work, which comprises fifty PET scans of lung, head and neck, and brain tumours. The results have confirmed that a benefit is indeed achieved in practice for all investigated anatomical districts, both quantitatively, through a set of commonly used quality indicators (dice similarity coefficient >87.66%, Hausdorff distance < 1.48 voxel and Mahalanobis distance < 0.82 voxel), and qualitatively in terms of Likert score (>3 in 54% of the tumours).

Gradient-based generation of intermediate images for heterogeneous tumor segmentation within hybrid PET/MRI scans

Segmentation of tumors from hybrid PET/MRI scans plays an essential role in accurate diagnosis and treatment planning. However, when treating tumors, several challenges, notably heterogeneity and the problem of leaking into surrounding tissues with similar high uptake, have to be considered. To address these issues, we propose an automated method for accurate delineation of tumors in hybrid PET/MRI scans. The method is mainly based on creating intermediate images. In fact, an automatic detection technique that determines a preliminary Interesting Uptake Region (IUR) is firstly performed. To overcome the leakage problem, a separation technique is adopted to generate the final IUR. Then, smart seeds are provided for the Graph Cut (GC) technique to obtain the tumor map. To create intermediate images that tend to reduce heterogeneity faced on the original images, the tumor map gradient is combined with the gradient image. Lastly, segmentation based on the GC_sum^max technique is applied to the generated images. The proposed method has been validated on PET phantoms as well as on real-world PET/MRI scans of prostate, liver and pancreatic tumors. Experimental comparison revealed the superiority of the proposed method over state-of-the-art methods. This confirms the crucial role of automatically creating intermediate images in addressing the problem of wrongly estimating arc weights for heterogeneous targets.

LinSEM: Linearizing segmentation evaluation metrics for medical images

Numerous algorithms are available for segmenting medical images. Empirical discrepancy metrics are commonly used in measuring the similarity or difference between segmentations by algorithms and "true" segmentations. However, one issue with the commonly used metrics is that the same metric value often represents different levels of "clinical acceptability" for different objects depending on their size, shape, and complexity of form. An ideal segmentation evaluation metric should be able to reflect degrees of acceptability directly from metric values and be able to show the same acceptability meaning by the same metric value for objects of different shape, size, and form. Intuitively, metrics which have a linear relationship with degree of acceptability will satisfy these conditions of the ideal metric. This issue has not been addressed in the medical image segmentation literature. In this paper, we propose a method called LinSEM for linearizing commonly used segmentation evaluation metrics based on corresponding degrees of acceptability evaluated by an expert in a reader study. LinSEM consists of two main parts: (a) estimating the relationship between metric values and degrees of acceptability separately for each considered metric and object, and (b) linearizing any given metric value corresponding to a given segmentation of an object based on the estimated relationship. Since algorithmic segmentations do not usually cover the full range of variability of acceptability, we create a set (SS) of simulated segmentations for each object that guarantee such coverage by using image transformations applied to a set (ST) of true segmentations of the object. We then conduct a reader study wherein the reader assigns an acceptability score (AS) for each sample in SS, expressing the acceptability of the sample on a 1 to 5 scale. Then the metric-AS relationship is constructed for the object by using an estimation method. With the idea that the ideal metric should be linear with respect to acceptability, we can then linearize the metric value of any segmentation sample of the object from a set (SA) of actual segmentations to its linearized value by using the constructed metric-acceptability relationship curve. Experiments are conducted involving three metrics - Dice coefficient (DC), Jaccard index (JI), and Hausdorff Distance (HD) - on five objects: skin outer boundary of the head and neck (cervico-thoracic) body region superior to the shoulders, right parotid gland, mandible, cervical esophagus, and heart. Actual segmentations (SA) of these objects are generated via our Automatic Anatomy Recognition (AAR) method. Our results indicate that, generally, JI has a more linear relationship with acceptability before linearization than other metrics. LinSEM achieves significantly improved uniformity of meaning post-linearization across all tested objects and metrics, except in a few cases where the departure from linearity was insignificant. This improvement is generally the largest for DC and HD reaching 8-25% for many tested cases. Although some objects (such as right parotid gland and esophagus for DC and JI) are close in their meaning between themselves before linearization, they are distant in this meaning from other objects but are brought close to other objects after linearization. This suggests the importance of performing linearization considering all objects in a body region and body-wide.

A User Interface for Optimizing Radiologist Engagement in Image Data Curation for Artificial Intelligence

Purpose

To delineate image data curation needs and describe a locally designed graphical user interface (GUI) to aid radiologists in image annotation for artificial intelligence (AI) applications in medical imaging.

Materials and Methods

GUI components support image analysis toolboxes, picture archiving and communication system integration, third-party applications, processing of scripting languages, and integration of deep learning libraries. For clinical AI applications, GUI components included two-dimensional segmentation and classification; three-dimensional segmentation and quantification; and three-dimensional segmentation, quantification, and classification. To assess radiologist engagement and performance efficiency associated with GUI-related capabilities, image annotation rate (studies per day) and speed (minutes per case) were evaluated in two clinical scenarios of varying complexity: hip fracture detection and coronary atherosclerotic plaque demarcation and stenosis grading.

Results

For hip fracture, 1050 radiographs were annotated over 7 days (150 studies per day; median speed: 10 seconds per study [interquartile range, 3-21 seconds per study]). A total of 294 coronary CT angiographic studies with 1843 arteries and branches were annotated for atherosclerotic plaque over 23 days (15.2 studies [80.1 vessels] per day; median speed: 6.08 minutes per study [interquartile range, 2.8-10.6 minutes per study] and 73 seconds per vessel [interquartile range, 20.9-155 seconds per vessel]).

Conclusion

GUI-component compatibility with common image analysis tools facilitates radiologist engagement in image data curation, including image annotation, supporting AI application development and evolution for medical imaging. When complemented by other GUI elements, a continuous integrated workflow supporting formation of an agile deep neural network life cycle results.Supplemental material is available for this article.© RSNA, 2019.

A novel automated lumen segmentation and classification algorithm for detection of irregular protrusion after stents deployment

BACKGROUND

Clinically, irregular protrusions and blockages after stent deployment can lead to significant adverse outcomes such as thrombotic reocclusion or restenosis. In this study, we propose a novel fully automated method for irregular lumen segmentation and normal/abnormal lumen classification.

METHODS

The proposed method consists of a lumen segmentation, feature extraction, and lumen classification. In total, 92 features were extracted to classify normal/abnormal lumen. The lumen classification method is a combination of supervised learning algorithm and feature selection that is a partition-membership filter method.

RESULTS

As the results, our proposed lumen segmentation method obtained the average of dice similarity coefficient (DSC) and the accuracy of proposed features and the random forest (RF) for normal/abnormal lumen classification as 97.6% and 98.2%, respectively.

CONCLUSIONS

Therefore, we can lead to better understanding of the overall vascular status and help to determine cardiovascular diagnosis.

Double-branched and area-constraint fully convolutional networks for automated serous retinal detachment segmentation in SD-OCT images

BACKGROUND AND OBJECTIVE

Quantitative assessment of subretinal fluid in spectral domain optical coherence tomography (SD-OCT) images is crucial for the diagnosis of central serous chorioretinopathy. For the subretinal fluid segmentation, the traditional methods need to segment retinal layers and then segment subretinal fluid. The layer segmentation has a high influence on subretinal fluid segmentation, so we aim to develop a deep learning model to segment subretinal fluid automatically without layer segmentation.

METHODS

In this paper, we propose a novel image-to-image double-branched and area-constraint fully convolutional networks (DA-FCN) for segmenting subretinal fluid in SD-OCT images. Firstly, the dataset is extended by mirroring image, which helps to overcome the over-fitting problem in the training stage. Then, double-branched structures are designed to learn the shallow coarse and deep representations from the SD-OCT images. DA-FCN model is directly trained using the image and corresponding pixel-based ground truth. Finally, we introduce a novel supervision mechanism by jointing the area loss L_A with the softmax loss L_S to learn more representative features.

RESULTS

The testing dataset with 52 SD-OCT volumes from 35 eyes of 35 patients is used for the evaluation of the proposed algorithm based on the cross-validation method. For the three criterions, including the true positive volume fraction, dice similarity coefficient, and positive predicative value, our method can obtain the results of (1) 94.3, 95.3, and 96.4 for dataset 1; (2) 97.3, 95.3, and 93.4 for dataset 2; (3) 93.0, 92.8, and 92.8 for dataset 3; (4) 89.7, 90.1, and 92.6 for dataset 4.

CONCLUSION

In this work, we propose a novel fully convolutional network for the automatic segmentation of the subretinal fluid. By constructing the double branched structures and area constraint term, our method shows higher segmentation accuracy without layer segmentation compared with other methods.

Automated segmentation of macular edema in OCT using deep neural networks

Macular edema is an eye disease that can affect visual acuity. Typical disease symptoms include subretinal fluid (SRF) and pigment epithelium detachment (PED). Optical coherence tomography (OCT) has been widely used for diagnosing macular edema because of its non-invasive and high resolution properties. Segmentation for macular edema lesions from OCT images plays an important role in clinical diagnosis. Many computer-aided systems have been proposed for the segmentation. Most traditional segmentation methods used in these systems are based on low-level hand-crafted features, which require significant domain knowledge and are sensitive to the variations of lesions. To overcome these shortcomings, this paper proposes to use deep neural networks (DNNs) together with atrous spatial pyramid pooling (ASPP) to automatically segment the SRF and PED lesions. Lesions-related features are first extracted by DNNs, then processed by ASPP which is composed of multiple atrous convolutions with different fields of view to accommodate the various scales of the lesions. Based on ASPP, a novel module called stochastic ASPP (sASPP) is proposed to combat the co-adaptation of multiple atrous convolutions. A large OCT dataset provided by a competition platform called "AI Challenger" are used to train and evaluate the proposed model. Experimental results demonstrate that the DNNs together with ASPP achieve higher segmentation accuracy compared with the state-of-the-art method. The stochastic operation added in sASPP is empirically verified as an effective regularization method that can alleviate the overfitting problem and significantly reduce the validation error.

A review on lung boundary detection in chest X-rays

PURPOSE

Chest radiography is the most common imaging modality for pulmonary diseases. Due to its wide usage, there is a rich literature addressing automated detection of cardiopulmonary diseases in digital chest X-rays (CXRs). One of the essential steps for automated analysis of CXRs is localizing the relevant region of interest, i.e., isolating lung region from other less relevant parts, for applying decision-making algorithms there. This article provides an overview of the recent literature on lung boundary detection in CXR images.

METHODS

We review the leading lung segmentation algorithms proposed in period 2006-2017. First, we present a review of articles for posterior-anterior view CXRs. Then, we mention studies which operate on lateral views. We pay particular attention to works that focus their efforts on deformed lungs and pediatric cases. We also highlight the radiographic measures extracted from lung boundary and their use in automatically detecting cardiopulmonary abnormalities. Finally, we identify challenges in dataset curation and expert delineation process, and we listed publicly available CXR datasets.

RESULTS

(1) We classified algorithms into four categories: rule-based, pixel classification-based, model-based, hybrid, and deep learning-based algorithms. Based on the reviewed articles, hybrid methods and deep learning-based methods surpass the algorithms in other classes and have segmentation performance as good as inter-observer performance. However, they require long training process and pose high computational complexity. (2) We found that most of the algorithms in the literature are evaluated on posterior-anterior view adult CXRs with a healthy lung anatomy appearance without considering challenges in abnormal CXRs. (3) We also found that there are limited studies for pediatric CXRs. The lung appearance in pediatrics, especially in infant cases, deviates from adult lung appearance due to the pediatric development stages. Moreover, pediatric CXRs are noisier than adult CXRs due to interference by other objects, such as someone holding the child's arms or the child's body, and irregular body pose. Therefore, lung boundary detection algorithms developed on adult CXRs may not perform accurately in pediatric cases and need additional constraints suitable for pediatric CXR imaging characteristics. (4) We have also stated that one of the main challenges in medical image analysis is accessing the suitable datasets. We listed benchmark CXR datasets for developing and evaluating the lung boundary algorithms. However, the number of CXR images with reference boundaries is limited due to the cumbersome but necessary process of expert boundary delineation.

CONCLUSIONS

A reliable computer-aided diagnosis system would need to support a greater variety of lung and background appearance. To our knowledge, algorithms in the literature are evaluated on posterior-anterior view adult CXRs with a healthy lung anatomy appearance, without considering ambiguous lung silhouettes due to pathological deformities, anatomical alterations due to misaligned body positioning, patient's development stage and gross background noises such as holding hands, jewelry, patient's head and legs in CXR. Considering all the challenges which are not very well addressed in the literature, developing lung boundary detection algorithms that are robust to such interference remains a challenging task. We believe that a broad review of lung region detection algorithms would be useful for researchers working in the field of automated detection/diagnosis algorithms for lung/heart pathologies in CXRs.

Deep Learning for Automated Segmentation of Liver Lesions at CT in Patients with Colorectal Cancer Liver Metastases

Purpose

To evaluate the performance, agreement, and efficiency of a fully convolutional network (FCN) for liver lesion detection and segmentation at CT examinations in patients with colorectal liver metastases (CLMs).

Materials and Methods

This retrospective study evaluated an automated method using an FCN that was trained, validated, and tested with 115, 15, and 26 contrast material-enhanced CT examinations containing 261, 22, and 105 lesions, respectively. Manual detection and segmentation by a radiologist was the reference standard. Performance of fully automated and user-corrected segmentations was compared with that of manual segmentations. The interuser agreement and interaction time of manual and user-corrected segmentations were assessed. Analyses included sensitivity and positive predictive value of detection, segmentation accuracy, Cohen κ, Bland-Altman analyses, and analysis of variance.

Results

In the test cohort, for lesion size smaller than 10 mm (n = 30), 10-20 mm (n = 35), and larger than 20 mm (n = 40), the detection sensitivity of the automated method was 10%, 71%, and 85%; positive predictive value was 25%, 83%, and 94%; Dice similarity coefficient was 0.14, 0.53, and 0.68; maximum symmetric surface distance was 5.2, 6.0, and 10.4 mm; and average symmetric surface distance was 2.7, 1.7, and 2.8 mm, respectively. For manual and user-corrected segmentation, κ values were 0.42 (95% confidence interval: 0.24, 0.63) and 0.52 (95% confidence interval: 0.36, 0.72); normalized interreader agreement for lesion volume was -0.10 ± 0.07 (95% confidence interval) and -0.10 ± 0.08; and mean interaction time was 7.7 minutes ± 2.4 (standard deviation) and 4.8 minutes ± 2.1 (P < .001), respectively.

Conclusion

Automated detection and segmentation of CLM by using deep learning with convolutional neural networks, when manually corrected, improved efficiency but did not substantially change agreement on volumetric measurements.© RSNA, 2019Supplemental material is available for this article.

Quantification of body-torso-wide tissue composition on low-dose CT images via automatic anatomy recognition

PURPOSE

Quantification of body composition plays an important role in many clinical and research applications. Radiologic imaging techniques such as Dual-energy X-ray absorptiometry (DXA), magnetic resonance imaging (MRI), and computed tomography (CT) imaging make accurate quantification of the body composition possible. However, most current imaging-based methods need human interaction to quantify multiple tissues. When dealing with whole-body images of many subjects, interactive methods become impractical. This paper presents an automated, efficient, accurate, and practical body composition quantification method for low-dose CT images.

METHOD

Our method, named automatic anatomy recognition body composition analysis (AAR-BCA), aims to quantify four tissue components in body torso (BT) - subcutaneous adipose tissue (SAT), visceral adipose tissue (VAT), bone tissue, and muscle tissue - from CT images of given whole-body positron emission tomography/computed tomography (PET/CT) acquisitions. AAR-BCA consists of three key steps - modeling BT with its ensemble of key objects from a population of patient images, recognition or localization of these objects in a given patient image I, and delineation and quantification of the four tissue components in I guided by the recognized objects. In the first step, from a given set of patient images and the associated delineated objects, a fuzzy anatomy model of the key object ensemble, including anatomic organs, tissue regions, and tissue interfaces, is built where the objects are organized in a hierarchical order. The second step involves recognizing, or finding roughly the location of, each object in any given whole-body image I of a patient following the object hierarchy and guided by the built model. The third step makes use of this fuzzy localization information of the objects and the intensity distributions of the four tissue components, already learned and encoded in the model, to optimally delineate in a fuzzy manner and quantify these components. All parameters in our method are determined from training datasets.

RESULTS

Thirty-eight low-dose CT images from different subjects are tested in a fivefold cross-validation strategy for evaluating AAR-BCA with a 23-15 train-test dataset division. For BT, over all objects, AAR-BCA achieves a false-positive volume fraction (FPVF) of 3.7% and false-negative volume fraction (FNVF) of 3.8%. Notably, SAT achieves both a FPVF and FNVF under 3%. For bone tissue, it achieves a FPVF and a FNVF both under 3.5%. For VAT tissue, the FNVF of 4.8% is higher than for other objects and so also for muscle (4.7%). The level of accuracy for the four tissue components in individual body subregions mostly remains at the same level as for BT. The processing time required per patient image is under a minute.

CONCLUSIONS

Motivated by applications in cancer and systemic diseases, our goal in this paper was to seek a practical method for body composition quantification which is automated, accurate, and efficient, and works on BT in low-dose CT. The proposed AAR-BCA method toward this goal can quantify four tissue components including SAT, VAT, bone tissue, and muscle tissue in the body torso with under 5% overall error. All needed parameters can be automatically estimated from the training datasets.

Active contour algorithm with discriminant analysis for delineating tumors in positron emission tomography

In the context of cancer delineation using positron emission tomography datasets, we present an innovative approach which purpose is to tackle the real-time, three-dimensional segmentation task in a full, or at least nearly full automatized way. The approach comprises a preliminary initialization phase where the user highlights a region of interest around the cancer on just one slice of the tomographic dataset. The algorithm takes care of identifying an optimal and user-independent region of interest around the anomalous tissue and located on the slice containing the highest standardized uptake value so to start the successive segmentation task. The three-dimensional volume is then reconstructed using a slice-by-slice marching approach until a suitable automatic stop condition is met. On each slice, the segmentation is performed using an enhanced local active contour based on the minimization of a novel energy functional which combines the information provided by a machine learning component, the discriminant analysis in the present study. As a result, the whole algorithm is almost completely automatic and the output segmentation is independent from the input provided by the user. Phantom experiments comprising spheres and zeolites, and clinical cases comprising various body districts (lung, brain, and head and neck), and two different radio-tracers (18 F-fluoro-2-deoxy-d-glucose, and 11C-labeled Methionine) were used to assess the algorithm performances. Phantom experiments with spheres and with zeolites showed a dice similarity coefficient above 90% and 80%, respectively. Clinical cases showed high agreement with the gold standard (R² = 0.98). These results indicate that the proposed method can be efficiently applied in the clinical routine with potential benefit for the treatment response assessment, and targeting in radiotherapy.

A Comparison of Thresholding Methods for Forensic Reconstruction Studies Using Fluorescent Powder Proxies for Trace Materials

Image segmentation is a fundamental precursor to quantitative image analysis. At present, no standardised methodology exists for segmenting images of fluorescent proxies for trace evidence. Experiments evaluated (i) whether manual segmentation is reproducible within and between examiners (with three participants repeatedly tracing three images) (ii) whether manually defining a threshold level offers accurate and reproducible results (with 20 examiners segmenting 10 images), and (iii) whether a global thresholding algorithm might perform with similar accuracy, while offering improved reproducibility and efficiency (16 algorithms tested). Statistically significant differences were seen between examiners’ traced outputs. Manually thresholding produced good accuracy on average (within ±1% of the expected values), but poor reproducibility (with multiple outliers). Three algorithms (Yen, MaxEntropy, and RenyiEntropy) offered similar accuracy, with improved reproducibility and efficiency. Together, these findings suggest that appropriate algorithms could perform thresholding tasks as part of a robust workflow for reconstruction studies employing fluorescent proxies for trace evidence.

A smart and operator independent system to delineate tumours in Positron Emission Tomography scans

Positron Emission Tomography (PET) imaging has an enormous potential to improve radiation therapy treatment planning offering complementary functional information with respect to other anatomical imaging approaches. The aim of this study is to develop an operator independent, reliable, and clinically feasible system for biological tumour volume delineation from PET images. Under this design hypothesis, we combine several known approaches in an original way to deploy a system with a high level of automation. The proposed system automatically identifies the optimal region of interest around the tumour and performs a slice-by-slice marching local active contour segmentation. It automatically stops when a "cancer-free" slice is identified. User intervention is limited at drawing an initial rough contour around the cancer region. By design, the algorithm performs the segmentation minimizing any dependence from the initial input, so that the final result is extremely repeatable. To assess the performances under different conditions, our system is evaluated on a dataset comprising five synthetic experiments and fifty oncological lesions located in different anatomical regions (i.e. lung, head and neck, and brain) using PET studies with 18F-fluoro-2-deoxy-d-glucose and 11C-labeled Methionine radio-tracers. Results on synthetic lesions demonstrate enhanced performances when compared against the most common PET segmentation methods. In clinical cases, the proposed system produces accurate segmentations (average dice similarity coefficient: 85.36 ± 2.94%, 85.98 ± 3.40%, 88.02 ± 2.75% in the lung, head and neck, and brain district, respectively) with high agreement with the gold standard (determination coefficient R² = 0.98). We believe that the proposed system could be efficiently used in the everyday clinical routine as a medical decision tool, and to provide the clinicians with additional information, derived from PET, which can be of use in radiation therapy, treatment, and planning.

Individual muscle segmentation in MR images: A 3D propagation through 2D non-linear registration approaches

Manual and automated segmentation of individual muscles in magnetic resonance images have been recognized as challenging given the high variability of shapes between muscles and subjects and the discontinuity or lack of visible boundaries between muscles. In the present study, we proposed an original algorithm allowing a semi-automatic transversal propagation of manually-drawn masks. Our strategy was based on several ascending and descending non-linear registration approaches which is similar to the estimation of a Lagrangian trajectory applied to manual masks. Using several manually-segmented slices, we have evaluated our algorithm on the four muscles of the quadriceps femoris group. We mainly showed that our 3D propagated segmentation was very accurate with an averaged Dice similarity coefficient value higher than 0.91 for the minimal manual input of only two manually-segmented slices.

LEFMIS: locally-oriented evaluation framework for medical image segmentation algorithms

This article proposes a novel framework for the locally-oriented evaluation of segmentation algorithms (LEFMIS). The presented approach is robust and takes into account local inter/intra-observer variability and the anisotropy of medical images. What is more, the framework makes it possible to distinguish types of error locally. These features are crucial in the context of cancer image data. The proposed framework is based on use of the signed anisotropic Euclidean distance transform and the distance projection. It can be used easily in many different applications with or without additional expert outlines (both inter- and intra-observer variability). The performance of the proposed framework is depicted using both artificial and kidney cancer CT data with experts' manual outlines. In the article, in the case of artificial data, it is presented that the manual outlines dispersion is symmetric in relation to the truth border. The effectiveness of the selected segmentation algorithm was analysed in the context of kidney cancer using computed tomography data. For the calculated local inter-observer variability, 80.11% of the surface points generated by the kidney segmentation algorithm are within one expert outline standard deviation and 97.96% are within five. An error distribution shift in the direction of type I error equivalent was also observed. Finally, the significance of the local estimation of error type differences is presented. The article shows the greater usefulness and flexibility of the proposed framework in comparison to the state-of-the-art methods. The exemplary usage of the LEFMIS with or without inter-/intra-observer variability is also presented.

Unsupervised CT Lung Image Segmentation of a Mycobacterium Tuberculosis Infection Model

Tuberculosis (TB) is an infectious disease caused by Mycobacterium tuberculosis that produces pulmonary damage. Radiological imaging is the preferred technique for the assessment of TB longitudinal course. Computer-assisted identification of biomarkers eases the work of the radiologist by providing a quantitative assessment of disease. Lung segmentation is the step before biomarker extraction. In this study, we present an automatic procedure that enables robust segmentation of damaged lungs that have lesions attached to the parenchyma and are affected by respiratory movement artifacts in a Mycobacterium Tuberculosis infection model. Its main steps are the extraction of the healthy lung tissue and the airway tree followed by elimination of the fuzzy boundaries. Its performance was compared with respect to a segmentation obtained using: (1) a semi-automatic tool and (2) an approach based on fuzzy connectedness. A consensus segmentation resulting from the majority voting of three experts' annotations was considered our ground truth. The proposed approach improves the overlap indicators (Dice similarity coefficient, 94% ± 4%) and the surface similarity coefficients (Hausdorff distance, 8.64 mm ± 7.36 mm) in the majority of the most difficult-to-segment slices. Results indicate that the refined lung segmentations generated could facilitate the extraction of meaningful quantitative data on disease burden.