A U-Net Deep Learning Framework for High Performance Vessel Segmentation in Patients With Cerebrovascular Disease

Automated intracranial vessel segmentation of 4D flow MRI data in patients with atherosclerotic stenosis using a convolutional neural network

Introduction

Intracranial 4D flow MRI enables quantitative assessment of hemodynamics in patients with intracranial atherosclerotic disease (ICAD). However, quantitative assessments are still challenging due to the time-consuming vessel segmentation, especially in the presence of stenoses, which can often result in user variability. To improve the reproducibility and robustness as well as to accelerate data analysis, we developed an accurate, fully automated segmentation for stenosed intracranial vessels using deep learning.

Methods

154 dual-VENC 4D flow MRI scans (68 ICAD patients with stenosis, 86 healthy controls) were retrospectively selected. Manual segmentations were used as ground truth for training. For automated segmentation, deep learning was performed using a 3D U-Net. 20 randomly selected cases (10 controls, 10 patients) were separated and solely used for testing. Cross-sectional areas and flow parameters were determined in the Circle of Willis (CoW) and the sinuses. Furthermore, the flow conservation error was calculated. For statistical comparisons, Dice scores (DS), Hausdorff distance (HD), average symmetrical surface distance (ASSD), Bland-Altman analyses, and interclass correlations were computed using the manual segmentations from two independent observers as reference. Finally, three stenosis cases were analyzed in more detail by comparing the 4D flow-based segmentations with segmentations from black blood vessel wall imaging (VWI).

Results

Training of the network took approximately 10 h and the average automated segmentation time was 2.2 ± 1.0 s. No significant differences in segmentation performance relative to two independent observers were observed. For the controls, mean DS was 0.85 ± 0.03 for the CoW and 0.86 ± 0.06 for the sinuses. Mean HD was 7.2 ± 1.5 mm (CoW) and 6.6 ± 3.7 mm (sinuses). Mean ASSD was 0.15 ± 0.04 mm (CoW) and 0.22 ± 0.17 mm (sinuses). For the patients, the mean DS was 0.85 ± 0.04 (CoW) and 0.82 ± 0.07 (sinuses), the HD was 8.4 ± 3.1 mm (CoW) and 5.7 ± 1.9 mm (sinuses) and the mean ASSD was 0.22 ± 0.10 mm (CoW) and 0.22 ± 0.11 mm (sinuses). Small bias and limits of agreement were observed in both cohorts for the flow parameters. The assessment of the cross-sectional lumen areas in stenosed vessels revealed very good agreement (ICC: 0.93) with the VWI segmentation but a consistent overestimation (bias ± LOA: 28.1 ± 13.9%).

Discussion

Deep learning was successfully applied for fully automated segmentation of stenosed intracranial vasculatures using 4D flow MRI data. The statistical analysis of segmentation and flow metrics demonstrated very good agreement between the CNN and manual segmentation and good performance in stenosed vessels. To further improve the performance and generalization, more ICAD segmentations as well as other intracranial vascular pathologies will be considered in the future.

Automated detection and classification of coronary atherosclerotic plaques on coronary CT angiography using deep learning algorithm

Background

Coronary artery disease (CAD) is the leading cause of mortality worldwide. Recent advances in deep learning technology promise better diagnosis of CAD and improve assessment of CAD plaque buildup. The purpose of this study is to assess the performance of a deep learning algorithm in detecting and classifying coronary atherosclerotic plaques in coronary computed tomographic angiography (CCTA) images.

Methods

Between January 2019 and September 2020, CCTA images of 669 consecutive patients with suspected CAD from Nanjing Drum Tower Hospital Clinical College of Nanjing University of Chinese Medicine were included in this study. There were 106 patients included in the retrospective plaque detection analysis, which was evaluated by a deep learning algorithm and four independent physicians with varying clinical experience. Additionally, 563 patients were included in the analysis for plaque classification using the deep learning algorithm, and their results were compared with those of expert radiologists. Plaques were categorized as absent, calcified, non-calcified, or mixed.

Results

The deep learning algorithm exhibited higher sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy {92% [95% confidence interval (CI): 89.5-94.1%], 87% (95% CI: 84.2-88.5%), 79% (95% CI: 76.1-82.4%), 95% (95% CI: 93.4-96.3%), and 89% (95% CI: 86.9-90.0%)} compared to physicians with ≤5 years of clinical experience in CAD diagnosis for the detection of coronary plaques. The algorithm's overall sensitivity, specificity, PPV, NPV, accuracy, and Cohen's kappa for plaque classification were 94% (95% CI: 92.3-94.7%), 90% (95% CI: 88.8-90.3%), 70% (95% CI: 68.3-72.1%), 98% (95% CI: 97.8-98.5%), 90% (95% CI: 89.8-91.1%) and 0.74 (95% CI: 0.70-0.78), indicating strong performance.

Conclusions

The deep learning algorithm has demonstrated reliable and accurate detection and classification of coronary atherosclerotic plaques in CCTA images. It holds the potential to enhance the diagnostic capabilities of junior radiologists and junior intervention cardiologists in the CAD diagnosis, as well as to streamline the triage of patients with acute coronary symptoms.

Patient-specific cerebral 3D vessel model reconstruction using deep learning

Three-dimensional vessel model reconstruction from patient-specific magnetic resonance angiography (MRA) images often requires some manual maneuvers. This study aimed to establish the deep learning (DL)-based method for vessel model reconstruction. Time of flight MRA of 40 patients with internal carotid artery aneurysms was prepared, and three-dimensional vessel models were constructed using the threshold and region-growing method. Using those datasets, supervised deep learning using 2D U-net was performed to reconstruct 3D vessel models. The accuracy of the DL-based vessel segmentations was assessed using 20 MRA images outside the training dataset. The dice coefficient was used as the indicator of the model accuracy, and the blood flow simulation was performed using the DL-based vessel model. The created DL model could successfully reconstruct a three-dimensional model in all 60 cases. The dice coefficient in the test dataset was 0.859. Of note, the DL-generated model proved its efficacy even for large aneurysms (> 10 mm in their diameter). The reconstructed model was feasible in performing blood flow simulation to assist clinical decision-making. Our DL-based method could successfully reconstruct a three-dimensional vessel model with moderate accuracy. Future studies are warranted to exhibit that DL-based technology can promote medical image processing.

Hepatic and portal vein segmentation with dual-stream deep neural network

BACKGROUND

Liver lesions mainly occur inside the liver parenchyma, which are difficult to locate and have complicated relationships with essential vessels. Thus, preoperative planning is crucial for the resection of liver lesions. Accurate segmentation of the hepatic and portal veins (PVs) on computed tomography (CT) images is of great importance for preoperative planning. However, manually labeling the mask of vessels is laborious and time-consuming, and the labeling results of different clinicians are prone to inconsistencies. Hence, developing an automatic segmentation algorithm for hepatic and PVs on CT images has attracted the attention of researchers. Unfortunately, existing deep learning based automatic segmentation methods are prone to misclassifying peripheral vessels into wrong categories.

PURPOSE

This study aims to provide a fully automatic and robust semantic segmentation algorithm for hepatic and PVs, guiding subsequent preoperative planning. In addition, to address the deficiency of the public dataset for hepatic and PV segmentation, we revise the annotations of the Medical Segmentation Decathlon (MSD) hepatic vessel segmentation dataset and add the masks of the hepatic veins (HVs) and PVs.

METHODS

We proposed a structure with a dual-stream encoder combining convolution and Transformer block, named Dual-stream Hepatic Portal Vein segmentation Network, to extract local features and long-distance spatial information, thereby extracting anatomical information of hepatic and portal vein, avoiding misdivisions of adjacent peripheral vessels. Besides, a multi-scale feature fusion block based on dilated convolution is proposed to extract multi-scale features on expanded perception fields for local features, and a multi-level fusing attention module is introduced for efficient context information extraction. Paired t-test is conducted to evaluate the significant difference in dice between the proposed methods and the comparing methods.

RESULTS

Two datasets are constructed from the original MSD dataset. For each dataset, 50 cases are randomly selected for model evaluation in the scheme of 5-fold cross-validation. The results show that our method outperforms the state-of-the-art Convolutional Neural Network-based and transformer-based methods. Specifically, for the first dataset, our model reaches 0.815, 0.830, and 0.807 at overall dice, precision, and sensitivity. The dice of the hepatic and PVs are 0.835 and 0.796, which also exceed the numeric result of the comparing methods. Almost all the p-values of paired t-tests on the proposed approach and comparing approaches are smaller than 0.05. On the second dataset, the proposed algorithm achieves 0.749, 0.762, 0.726, 0.835, and 0.796 for overall dice, precision, sensitivity, dice for HV, and dice for PV, among which the first four numeric results exceed comparing methods.

CONCLUSIONS

The proposed method is effective in solving the problem of misclassifying interlaced peripheral veins for the HV and PV segmentation task and outperforming the comparing methods on the relabeled dataset.

FFCM-MRF: An accurate and generalizable cerebrovascular segmentation pipeline for humans and rhesus monkeys based on TOF-MRA

PURPOSE

Cerebrovascular segmentation and quantification of vascular morphological features in humans and rhesus monkeys are essential for prevention, diagnosis, and treatment of brain diseases. However, current automated whole-brain vessel segmentation methods are often not generalizable to independent datasets, limiting their usefulness in real-world environments with their heterogeneity in participants, scanners, and species.

MATERIALS AND METHODS

In this study, we proposed an automated, accurate and generalizable segmentation method for magnetic resonance angiography images called FFCM-MRF. This method integrated fast fuzzy c-means clustering and Markov random field optimization by vessel shape priors and spatial constraints. We used a total of 123 human and 44 macaque MRA images scanned at 1.5 T, 3 T, and 7 T MRI from 9 datasets to develop and validate the method.

RESULTS

FFCM-MRF achieved average Dice similarity coefficients ranging from 69.16 % to 89.63 % across multiple independent datasets, with improvements ranging from 3.24 % to 7.3 % compared to state-of-the-art methods. Quantitative analysis showed that FFCM-MRF can accurately segment major arteries in the Circle of Willis at the base of the brain and small distal pial arteries while effectively reducing noise. Test-retest analysis showed that the model yielded high vascular volume and diameter reliability.

CONCLUSIONS

Our results have demonstrated that FFCM-MRF is highly accurate and reliable and largely independent of variations in field strength, scanner platforms, acquisition parameters, and species. The macaque MRA data and user-friendly open-source toolbox are freely available at OpenNeuro and GitHub to facilitate studies of imaging biomarkers for cerebrovascular and neurodegenerative diseases.

Deep learning-enabled detection of hypoxic-ischemic encephalopathy after cardiac arrest in CT scans: a comparative study of 2D and 3D approaches

Objective

To establish a deep learning model for the detection of hypoxic-ischemic encephalopathy (HIE) features on CT scans and to compare various networks to determine the best input data format.

Methods

168 head CT scans of patients after cardiac arrest were retrospectively identified and classified into two categories: 88 (52.4%) with radiological evidence of severe HIE and 80 (47.6%) without signs of HIE. These images were randomly divided into a training and a test set, and five deep learning models based on based on Densely Connected Convolutional Networks (DenseNet121) were trained and validated using different image input formats (2D and 3D images).

Results

All optimized stacked 2D and 3D networks could detect signs of HIE. The networks based on the data as 2D image data stacks provided the best results (S100: AUC: 94%, ACC: 79%, S50: AUC: 93%, ACC: 79%). We provide visual explainability data for the decision making of our AI model using Gradient-weighted Class Activation Mapping.

Conclusion

Our proof-of-concept deep learning model can accurately identify signs of HIE on CT images. Comparing different 2D- and 3D-based approaches, most promising results were achieved by 2D image stack models. After further clinical validation, a deep learning model of HIE detection based on CT images could be implemented in clinical routine and thus aid clinicians in characterizing imaging data and predicting outcome.

TL-MSE2-Net: Transfer learning based nested model for cerebrovascular segmentation with aneurysms

Cerebrovascular (i.e., cerebral vessel) segmentation is essential for diagnosing and treating brain diseases. Convolutional neural network models, such as U-Net, are commonly used for this purpose. Unfortunately, such models may not be entirely satisfactory in dealing with cerebrovascular segmentation with tumors due to the following issues: (1) Relatively small number of clinical datasets from patients obtained through different modalities such as computed tomography (CT) and magnetic resonance imaging (MRI), leading to inadequate training and lack of transferability in the modeling; (2) Insufficient feature extraction caused by less attention to both convolution sizes and cerebral vessel edges. Inspired by the existence of similar features on cerebral vessels between normal subjects and patients, we propose a transfer learning strategy based on a pre-trained nested model called TL-MSE2-Net. This model uses one of the publicly available datasets for cerebrovascular segmentation with aneurysms. To address issue (1), our transfer learning strategy leverages a pre-trained model that uses a large number of datasets from normal subjects, providing a potential solution to the lack of sufficient clinical datasets. To tackle issue (2), we structure the pre-trained model based on 3D U-Net, comprising three blocks: ResMul, DeRes, and REAM. The ResMul and DeRes blocks enhance feature extraction by utilizing multiple convolution sizes to capture multiscale features, and the REAM block increases the weight of the voxels on the edges of the given 3D volume. We evaluated the proposed model on one small private clinical dataset and two publicly available datasets. The experimental results demonstrated that our MSE2-Net framework achieved an average Dice score of 70.81 % and 89.08 % on the two publicly available datasets, outperforming other state-of-the-art methods. Ablation studies were also conducted to validate the effectiveness of each block. The proposed TL-MSE2-Net yielded better results than MSE2-Net on a small private clinical dataset, with increases of 5.52 %, 3.37 %, 6.71 %, and 0.85 % for the Dice score, sensitivity, Jaccard index, and precision, respectively.

TaG-Net: Topology-Aware Graph Network for Centerline-Based Vessel Labeling

Anatomical labeling of head and neck vessels is a vital step for cerebrovascular disease diagnosis. However, it remains challenging to automatically and accurately label vessels in computed tomography angiography (CTA) since head and neck vessels are tortuous, branched, and often spatially close to nearby vasculature. To address these challenges, we propose a novel topology-aware graph network (TaG-Net) for vessel labeling. It combines the advantages of volumetric image segmentation in the voxel space and centerline labeling in the line space, wherein the voxel space provides detailed local appearance information, and line space offers high-level anatomical and topological information of vessels through the vascular graph constructed from centerlines. First, we extract centerlines from the initial vessel segmentation and construct a vascular graph from them. Then, we conduct vascular graph labeling using TaG-Net, in which techniques of topology-preserving sampling, topology-aware feature grouping, and multi-scale vascular graph are designed. After that, the labeled vascular graph is utilized to improve volumetric segmentation via vessel completion. Finally, the head and neck vessels of 18 segments are labeled by assigning centerline labels to the refined segmentation. We have conducted experiments on CTA images of 401 subjects, and experimental results show superior vessel segmentation and labeling of our method compared to other state-of-the-art methods.

Using deep learning for an automatic detection and classification of the vascular bifurcations along the Circle of Willis

Most of the intracranial aneurysms (ICA) occur on a specific portion of the cerebral vascular tree named the Circle of Willis (CoW). More particularly, they mainly arise onto fifteen of the major arterial bifurcations constituting this circular structure. Hence, for an efficient and timely diagnosis it is critical to develop some methods being able to accurately recognize each Bifurcation of Interest (BoI). Indeed, an automatic extraction of the bifurcations presenting the higher risk of developing an ICA would offer the neuroradiologists a quick glance at the most alarming areas. Due to the recent efforts on Artificial Intelligence, Deep Learning turned out to be the best performing technology for many pattern recognition tasks. Moreover, various methods have been particularly designed for medical image analysis purposes. This study intends to assist the neuroradiologists to promptly locate any bifurcation presenting a high risk of ICA occurrence. It can be seen as a Computer Aided Diagnosis scheme, where the Artificial Intelligence facilitates the access to the regions of interest within the MRI. In this work, we propose a method for a fully automatic detection and recognition of the bifurcations of interest forming the Circle of Willis. Several neural networks architectures have been tested, and we thoroughly evaluate the bifurcation recognition rate.

Modeling and hexahedral meshing of cerebral arterial networks from centerlines

Computational fluid dynamics (CFD) simulation provides valuable information on blood flow from the vascular geometry. However, it requires extracting precise models of arteries from low-resolution medical images, which remains challenging. Centerline-based representation is widely used to model large vascular networks with small vessels, as it encodes both the geometric and topological information and facilitates manual editing. In this work, we propose an automatic method to generate a structured hexahedral mesh suitable for CFD directly from centerlines. We addressed both the modeling and meshing tasks. We proposed a vessel model based on penalized splines to overcome the limitations inherent to the centerline representation, such as noise and sparsity. The bifurcations are reconstructed using a parametric model based on the anatomy that we extended to planar n-furcations. Finally, we developed a method to produce a volume mesh with structured, hexahedral, and flow-oriented cells from the proposed vascular network model. The proposed method offers better robustness to the common defects of centerlines and increases the mesh quality compared to state-of-the-art methods. As it relies on centerlines alone, it can be applied to edit the vascular model effortlessly to study the impact of vascular geometry and topology on hemodynamics. We demonstrate the efficiency of our method by entirely meshing a dataset of 60 cerebral vascular networks. 92% of the vessels and 83% of the bifurcations were meshed without defects needing manual intervention, despite the challenging aspect of the input data. The source code is released publicly.

Seasonal pigment fluctuation in diploid and polyploid Arabidopsis revealed by machine learning-based phenotyping method PlantServation

Long-term field monitoring of leaf pigment content is informative for understanding plant responses to environments distinct from regulated chambers but is impractical by conventional destructive measurements. We developed PlantServation, a method incorporating robust image-acquisition hardware and deep learning-based software that extracts leaf color by detecting plant individuals automatically. As a case study, we applied PlantServation to examine environmental and genotypic effects on the pigment anthocyanin content estimated from leaf color. We processed >4 million images of small individuals of four Arabidopsis species in the field, where the plant shape, color, and background vary over months. Past radiation, coldness, and precipitation significantly affected the anthocyanin content. The synthetic allopolyploid A. kamchatica recapitulated the fluctuations of natural polyploids by integrating diploid responses. The data support a long-standing hypothesis stating that allopolyploids can inherit and combine the traits of progenitors. PlantServation facilitates the study of plant responses to complex environments termed "in natura".

Personalised simulation of hemodynamics in cerebrovascular disease: lessons learned from a study of diagnostic accuracy

Intracranial atherosclerotic disease (ICAD) poses a significant risk of subsequent stroke but current prevention strategies are limited. Mechanistic simulations of brain hemodynamics offer an alternative precision medicine approach by utilising individual patient characteristics. For clinical use, however, current simulation frameworks have insufficient validation. In this study, we performed the first quantitative validation of a simulation-based precision medicine framework to assess cerebral hemodynamics in patients with ICAD against clinical standard perfusion imaging. In a retrospective analysis, we used a 0-dimensional simulation model to detect brain areas that are hemodynamically vulnerable to subsequent stroke. The main outcome measures were sensitivity, specificity, and area under the receiver operating characteristics curve (ROC AUC) of the simulation to identify brain areas vulnerable to subsequent stroke as defined by quantitative measurements of relative mean transit time (relMTT) from dynamic susceptibility contrast MRI (DSC-MRI). In 68 subjects with unilateral stenosis >70% of the internal carotid artery (ICA) or middle cerebral artery (MCA), the sensitivity and specificity of the simulation were 0.65 and 0.67, respectively. The ROC AUC was 0.68. The low-to-moderate accuracy of the simulation may be attributed to assumptions of Newtonian blood flow, rigid vessel walls, and the use of time-of-flight MRI for geometric representation of subject vasculature. Future simulation approaches should focus on integrating additional patient data, increasing accessibility of precision medicine tools to clinicians, addressing disease burden disparities amongst different populations, and quantifying patient benefit. Our results underscore the need for further improvement of mechanistic simulations of brain hemodynamics to foster the translation of the technology to clinical practice.

A three-dimensional method for morphological analysis and flow velocity estimation in microvasculature on-a-chip

Three-dimensional (3D) imaging techniques (e.g., confocal microscopy) are commonly used to visualize in vitro models, especially microvasculature on-a-chip. Conversely, 3D analysis is not the standard method to extract quantitative information from those models. We developed the μVES algorithm to analyze vascularized in vitro models leveraging 3D data. It computes morphological parameters (geometry, diameter, length, tortuosity, eccentricity) and intravascular flow velocity. μVES application to microfluidic vascularized in vitro models shows that they successfully replicate functional features of the microvasculature in vivo in terms of intravascular fluid flow velocity. However, wall shear stress is lower compared to in vivo references. The morphological analysis also highlights the model's physiological similarities (vessel length and tortuosity) and shortcomings (vessel radius and surface-over-volume ratio). The addition of the third dimension in our analysis produced significant differences in the metrics assessed compared to 2D estimations. It enabled the computation of new indices, such as vessel eccentricity. These μVES capabilities can find application in analyses of different in vitro vascular models, as well as in vivo and ex vivo microvasculature.

Tubular Structure Segmentation via Multi-Scale Reverse Attention Sparse Convolution

Cerebrovascular and airway structures are tubular structures used for transporting blood and gases, respectively, providing essential support for the normal activities of the human body. Accurately segmenting these tubular structures is the basis of morphology research and pathological detection. Nevertheless, accurately segmenting these structures from images presents great challenges due to their complex morphological and topological characteristics. To address this challenge, this paper proposes a framework UARAI based on the U-Net multi-scale reverse attention network and sparse convolution network. The framework utilizes a multi-scale structure to effectively extract the global and deep detail features of vessels and airways. Further, it enhances the extraction ability of fine-edged features by a joint reverse attention module. In addition, the sparse convolution structure is introduced to improve the features' expression ability without increasing the model's complexity. Finally, the proposed training sample cropping strategy reduces the influence of block boundaries on the accuracy of tubular structure segmentation. The experimental findings demonstrate that the UARAI-based metrics, namely Dice and IoU, achieve impressive scores of 90.31% and 82.33% for cerebrovascular segmentation and 93.34% and 87.51% for airway segmentation, respectively. Compared to commonly employed segmentation techniques, the proposed method exhibits remarkable accuracy and robustness in delineating tubular structures such as cerebrovascular and airway structures. These results hold significant promise in facilitating medical image analysis and clinical diagnosis, offering invaluable support to healthcare professionals.

Digital labeling for 3D histology: segmenting blood vessels without a vascular contrast agent using deep learning

Recent advances in optical tissue clearing and three-dimensional (3D) fluorescence microscopy have enabled high resolution in situ imaging of intact tissues. Using simply prepared samples, we demonstrate here "digital labeling," a method to segment blood vessels in 3D volumes solely based on the autofluorescence signal and a nuclei stain (DAPI). We trained a deep-learning neural network based on the U-net architecture using a regression loss instead of a commonly used segmentation loss to achieve better detection of small vessels. We achieved high vessel detection accuracy and obtained accurate vascular morphometrics such as vessel length density and orientation. In the future, such digital labeling approach could easily be transferred to other biological structures.

Bidirectional meta-Kronecker factored optimizer and Hausdorff distance loss for few-shot medical image segmentation

To increase the accuracy of medical image analysis using supervised learning-based AI technology, a large amount of accurately labeled training data is required. However, the supervised learning approach may not be applicable to real-world medical imaging due to the lack of labeled data, the privacy of patients, and the cost of specialized knowledge. To handle these issues, we utilized Kronecker-factored decomposition, which enhances both computational efficiency and stability of the learning process. We combined this approach with a model-agnostic meta-learning framework for the parameter optimization. Based on this method, we present a bidirectional meta-Kronecker factored optimizer (BM-KFO) framework to quickly optimize semantic segmentation tasks using just a few magnetic resonance imaging (MRI) images as input. This model-agnostic approach can be implemented without altering network components and is capable of learning the learning process and meta-initial points while training on previously unseen data. We also incorporated a combination of average Hausdorff distance loss (AHD-loss) and cross-entropy loss into our objective function to specifically target the morphology of organs or lesions in medical images. Through evaluation of the proposed method on the abdominal MRI dataset, we obtained an average performance of 78.07% in setting 1 and 79.85% in setting 2. Our experiments demonstrate that BM-KFO with AHD-loss is suitable for general medical image segmentation applications and achieves superior performance compared to the baseline method in few-shot learning tasks. In order to replicate the proposed method, we have shared our code on GitHub. The corresponding URL can be found: https://github.com/YeongjoonKim/BMKFO.git .

Multiple sclerosis lesion segmentation: revisiting weighting mechanisms for federated learning

Background and introduction

Federated learning (FL) has been widely employed for medical image analysis to facilitate multi-client collaborative learning without sharing raw data. Despite great success, FL's applications remain suboptimal in neuroimage analysis tasks such as lesion segmentation in multiple sclerosis (MS), due to variance in lesion characteristics imparted by different scanners and acquisition parameters.

Methods

In this work, we propose the first FL MS lesion segmentation framework via two effective re-weighting mechanisms. Specifically, a learnable weight is assigned to each local node during the aggregation process, based on its segmentation performance. In addition, the segmentation loss function in each client is also re-weighted according to the lesion volume for the data during training.

Results

The proposed method has been validated on two FL MS segmentation scenarios using public and clinical datasets. Specifically, the case-wise and voxel-wise Dice score of the proposed method under the first public dataset is 65.20 and 74.30, respectively. On the second in-house dataset, the case-wise and voxel-wise Dice score is 53.66, and 62.31, respectively.

Discussions and conclusions

The Comparison experiments on two FL MS segmentation scenarios using public and clinical datasets have demonstrated the effectiveness of the proposed method by significantly outperforming other FL methods. Furthermore, the segmentation performance of FL incorporating our proposed aggregation mechanism can achieve comparable performance to that from centralized training with all the raw data.

ADSeg: A flap-attention-based deep learning approach for aortic dissection segmentation

Accurate and rapid segmentation of the lumen in an aortic dissection (AD) is an important prerequisite for risk evaluation and medical planning for patients with this serious condition. Although some recent studies have pioneered technical advances for the challenging AD segmentation task, they generally neglect the intimal flap structure that separates the true and false lumens. Identification and segmentation of the intimal flap may simplify AD segmentation, and the incorporation of long-distance z axis information interaction along the curved aorta may improve segmentation accuracy. This study proposes a flap attention module that focuses on key flap voxels and performs operations with long-distance attention. In addition, a pragmatic cascaded network structure with feature reuse and a two-step training strategy are presented to fully exploit network representation power. The proposed ADSeg method was evaluated on a multicenter dataset of 108 cases, with or without thrombus; ADSeg outperformed previous state-of-the-art methods by a significant margin and was robust against center variation.

White matter injury detection based on preterm infant cranial ultrasound images

Introduction

White matter injury (WMI) is now the major disease that seriously affects the quality of life of preterm infants and causes cerebral palsy of children, which also causes periventricular leuko-malacia (PVL) in severe cases. The study aimed to develop a method based on cranial ultrasound images to evaluate the risk of WMI.

Methods

This study proposed an ultrasound radiomics diagnostic system to predict the WMI risk. A multi-task deep learning model was used to segment white matter and predict the WMI risk simultaneously. In total, 158 preterm infants with 807 cranial ultrasound images were enrolled. WMI occurred in 32preterm infants (20.3%, 32/158).

Results

Ultrasound radiomics diagnostic system implemented a great result with AUC of 0.845 in the testing set. Meanwhile, multi-task deep learning model preformed a promising result both in segmentation of white matter with a Dice coefficient of 0.78 and prediction of WMI risk with AUC of 0.863 in the testing cohort.

Discussion

In this study, we presented a data-driven diagnostic system for white matter injury in preterm infants. The system combined multi-task deep learning and traditional radiomics features to achieve automatic detection of white matter regions on the one hand, and design a fusion strategy of deep learning features and manual radiomics features on the other hand to obtain stable and efficient diagnostic performance.

LIVE-Net: Comprehensive 3D vessel extraction framework in CT angiography

The extraction of vessels from computed tomography angiography (CTA) is significant in diagnosing and evaluating vascular diseases. However, due to the anatomical complexity, wide intensity distribution, and small volume proportion of vessels, vessel extraction is laborious and time-consuming, and it is easy to lead to error-prone diagnostic results in clinical practice. This study proposes a novel comprehensive vessel extraction framework, called the Local Iterative-based Vessel Extraction Network (LIVE-Net), to achieve 3D vessel segmentation while tracking vessel centerlines. LIVE-Net contains dual dataflow pathways that work alternately: an iterative tracking network and a local segmentation network. The former can generate the fine-grain direction and radius prediction of a vascular patch by using the attention-embedded atrous pyramid network (aAPN), and the latter can achieve 3D vascular lumen segmentation by constructing the multi-order self-attention U-shape network (MOSA-UNet). LIVE-Net is trained and evaluated on two datasets: the MICCAI 2008 Coronary Artery Tracking Challenge (CAT08) dataset and head and neck CTA dataset from the clinic. Experimental results of both tracking and segmentation show that our proposed LIVE-Net exhibits superior performance compared with other state-of-the-art (SOTA) networks. In the CAT08 dataset, the tracked centerlines have an average overlap of 95.2%, overlap until first error of 91.2%, overlap with the clinically relevant vessels of 98.3%, and error distance inside of 0.21 mm. The corresponding tracking overlap metrics in the head and neck CTA dataset are 96.7%, 91.0%, and 99.8%, respectively. In addition, the results of the consistent experiment also show strong clinical correspondence. For the segmentation of bilateral carotid and vertebral arteries, our method can not only achieve better accuracy with an average dice similarity coefficient (DSC) of 90.03%, Intersection over Union (IoU) of 81.97%, and 95% Hausdorff distance (95%HD) of 3.42 mm , but higher efficiency with an average time of 67.25 s , even three times faster compared to some methods applied in full field view. Both the tracking and segmentation results prove the potential clinical utility of our network.

A learnable Gabor Convolution kernel for vessel segmentation

Vessel segmentation is significant for characterizing vascular diseases, receiving wide attention of researchers. The common vessel segmentation methods are mainly based on convolutional neural networks (CNNs), which have excellent feature learning capabilities. Owing to inability to predict learning direction, CNNs generate large channels or sufficient depth to obtain sufficient features. It may engender redundant parameters. Drawing on performance ability of Gabor filters in vessel enhancement, we built Gabor convolution kernel and designed its optimization. Unlike traditional filter using and common modulation, its parameters are automatically updated using gradients in the back propagation. Since the structural shape of Gabor convolution kernels is the same as that of regular convolution kernels, it can be integrated into any CNNs architecture. We built Gabor ConvNet using Gabor convolution kernels and tested it using three vessel datasets. It scored 85.06%, 70.52% and 67.11%, respectively, ranking first on three datasets. Results shows that our method outperforms advanced models in vessel segmentation. Ablations also proved that Gabor kernel has better vessel extraction ability than the regular convolution kernel.

Hepatic vessels segmentation using deep learning and preprocessing enhancement

PURPOSE

Liver hepatic vessels segmentation is a crucial step for the diagnosis process in patients with hepatic diseases. Segmentation of liver vessels helps to study the liver internal segmental anatomy that helps in the preoperative planning of surgical treatment.

METHODS

Recently, the convolutional neural networks (CNN) have been proved to be efficient for the task of medical image segmentation. The paper proposes an automatic deep learning-based system for liver hepatic vessels segmentation of Computed Tomography (CT) datasets from different sources. The proposed work focuses on the combination of different steps; it starts by a preprocessing step to improve the vessels appearance within the liver region of interest in the CT scans. Coherence enhancing diffusion filtering (CED) and vesselness filtering methods are used to improve vessels contrast and intensity homogeneity. The proposed U-net based network architecture is implemented with modified residual block to include concatenation skip connection. The effect of enhancement using filtering step was studied. Also, the effect of data mismatch used in training and validation is studied.

RESULTS

The proposed method is evaluated using many CT datasets. Dice similarity coefficient (DSC) is used to evaluate the method. The average DSC score achieved a score 79%.

CONCLUSIONS

The proposed approach succeeded to segment liver vasculature from the liver envelope accurately, which makes it as potential tool for clinical preoperative planning.

Deep Learning for Head and Neck CT Angiography: Stenosis and Plaque Classification

Background Studies have rarely investigated stenosis detection from head and neck CT angiography scans because accurate interpretation is time consuming and labor intensive. Purpose To develop an automated convolutional neural network-based method for accurate stenosis detection and plaque classification in head and neck CT angiography images and compare its performance with that of radiologists. Materials and Methods A deep learning (DL) algorithm was constructed and trained with use of head and neck CT angiography images that were collected retrospectively from four tertiary hospitals between March 2020 and July 2021. CT scans were partitioned into training, validation, and independent test sets at a ratio of 7:2:1. An independent test set of CT angiography scans was collected prospectively between October 2021 and December 2021 in one of the four tertiary centers. Stenosis grade categories were as follows: mild stenosis (<50%), moderate stenosis (50%-69%), severe stenosis (70%-99%), and occlusion (100%). The stenosis diagnosis and plaque classification of the algorithm were compared with the ground truth of consensus by two radiologists (with more than 10 years of experience). The performance of the models was analyzed in terms of accuracy, sensitivity, specificity, and areas under the receiver operating characteristic curve. Results There were 3266 patients (mean age ± SD, 62 years ± 12; 2096 men) evaluated. The consistency between radiologists and the DL-assisted algorithm on plaque classification was 85.6% (320 of 374 cases [95% CI: 83.2, 88.6]) on a per-vessel basis. Moreover, the artificial intelligence model assisted in visual assessment, such as increasing confidence in the degree of stenosis. This reduced the time needed for diagnosis and report writing of radiologists from 28.8 minutes ± 5.6 to 12.4 minutes ± 2.0 (P < .001). Conclusion A deep learning algorithm for head and neck CT angiography interpretation accurately determined vessel stenosis and plaque classification and had equivalent diagnostic performance when compared with experienced radiologists. © RSNA, 2023 Supplemental material is available for this article.

High-throughput 3DRA segmentation of brain vasculature and aneurysms using deep learning

BACKGROUND AND OBJECTIVES

Automatic segmentation of the cerebral vasculature and aneurysms facilitates incidental detection of aneurysms. The assessment of aneurysm rupture risk assists with pre-operative treatment planning and enables in-silico investigation of cerebral hemodynamics within and in the vicinity of aneurysms. However, ensuring precise and robust segmentation of cerebral vessels and aneurysms in neuroimaging modalities such as three-dimensional rotational angiography (3DRA) is challenging. The vasculature constitutes a small proportion of the image volume, resulting in a large class imbalance (relative to surrounding brain tissue). Additionally, aneurysms and vessels have similar image/appearance characteristics, making it challenging to distinguish the aneurysm sac from the vessel lumen.

METHODS

We propose a novel multi-class convolutional neural network to tackle these challenges and facilitate the automatic segmentation of cerebral vessels and aneurysms in 3DRA images. The proposed model is trained and evaluated on an internal multi-center dataset and an external publicly available challenge dataset.

RESULTS

On the internal clinical dataset, our method consistently outperformed several state-of-the-art approaches for vessel and aneurysm segmentation, achieving an average Dice score of 0.81 (0.15 higher than nnUNet) and an average surface-to-surface error of 0.20 mm (less than the in-plane resolution (0.35 mm/pixel)) for aneurysm segmentation; and an average Dice score of 0.91 and average surface-to-surface error of 0.25 mm for vessel segmentation. In 223 cases of a clinical dataset, our method accurately segmented 190 aneurysm cases.

CONCLUSIONS

The proposed approach can help address class imbalance problems and inter-class interference problems in multi-class segmentation. Besides, this method performs consistently on clinical datasets from four different sources and the generated results are qualified for hemodynamic simulation. Code available at https://github.com/cistib/vessel-aneurysm-segmentation.

Generative Consistency for Semi-Supervised Cerebrovascular Segmentation From TOF-MRA

Cerebrovascular segmentation from Time-of-flight magnetic resonance angiography (TOF-MRA) is a critical step in computer-aided diagnosis. In recent years, deep learning models have proved its powerful feature extraction for cerebrovascular segmentation. However, they require many labeled datasets to implement effective driving, which are expensive and professional. In this paper, we propose a generative consistency for semi-supervised (GCS) model. Considering the rich information contained in the feature map, the GCS model utilizes the generation results to constrain the segmentation model. The generated data comes from labeled data, unlabeled data, and unlabeled data after perturbation, respectively. The GCS model also calculates the consistency of the perturbed data to improve the feature mining ability. Subsequently, we propose a new model as the backbone of the GSC model. It transfers TOF-MRA into graph space and establishes correlation using Transformer. We demonstrated the effectiveness of the proposed model on TOF-MRA representations, and tested the GCS model with state-of-the-art semi-supervised methods using the proposed model as backbone. The experiments prove the important role of the GCS model in cerebrovascular segmentation. Code is available at https://github.com/MontaEllis/SSL-For-Medical-Segmentation.

The U-Net Approaches to Evaluation of Dental Bite-Wing Radiographs: An Artificial Intelligence Study

Bite-wing radiographs are one of the most used intraoral radiography techniques in dentistry. AI is extremely important in terms of more efficient patient care in the field of dentistry. The aim of this study was to perform a diagnostic evaluation on bite-wing radiographs with an AI model based on CNNs. In this study, 500 bite-wing radiographs in the radiography archive of Eskişehir Osmangazi University, Faculty of Dentistry, Department of Oral and Maxillofacial Radiology were used. The CranioCatch labeling program (CranioCatch, Eskisehir, Turkey) with tooth decays, crowns, pulp, restoration material, and root-filling material for five different diagnoses were made by labeling the segmentation technique. The U-Net architecture was used to develop the AI model. F1 score, sensitivity, and precision results of the study, respectively, caries 0.8818-0.8235-0.9491, crown; 0.9629-0.9285-1, pulp; 0.9631-0.9843-0.9429, with restoration material; and 0.9714-0.9622-0.9807 was obtained as 0.9722-0.9459-1 for the root filling material. This study has shown that an AI model can be used to automatically evaluate bite-wing radiographs and the results are promising. Owing to these automatically prepared charts, physicians in a clinical intense tempo will be able to work more efficiently and quickly.

Challenges for machine learning in clinical translation of big data imaging studies

Combining deep learning image analysis methods and large-scale imaging datasets offers many opportunities to neuroscience imaging and epidemiology. However, despite these opportunities and the success of deep learning when applied to a range of neuroimaging tasks and domains, significant barriers continue to limit the impact of large-scale datasets and analysis tools. Here, we examine the main challenges and the approaches that have been explored to overcome them. We focus on issues relating to data availability, interpretability, evaluation, and logistical challenges and discuss the problems that still need to be tackled to enable the success of "big data" deep learning approaches beyond research.

Deep Learning Hybrid Techniques for Brain Tumor Segmentation

Medical images play an important role in medical diagnosis and treatment. Oncologists analyze images to determine the different characteristics of deadly diseases, plan the therapy, and observe the evolution of the disease. The objective of this paper is to propose a method for the detection of brain tumors. Brain tumors are identified from Magnetic Resonance (MR) images by performing suitable segmentation procedures. The latest technical literature concerning radiographic images of the brain shows that deep learning methods can be implemented to extract specific features of brain tumors, aiding clinical diagnosis. For this reason, most data scientists and AI researchers work on Machine Learning methods for designing automatic screening procedures. Indeed, an automated method would result in quicker segmentation findings, providing a robust output with respect to possible differences in data sources, mostly due to different procedures in data recording and storing, resulting in a more consistent identification of brain tumors. To improve the performance of the segmentation procedure, new architectures are proposed and tested in this paper. We propose deep neural networks for the detection of brain tumors, trained on the MRI scans of patients' brains. The proposed architectures are based on convolutional neural networks and inception modules for brain tumor segmentation. A comparison of these proposed architectures with the baseline reference ones shows very interesting results. MI-Unet showed a performance increase in comparison to baseline Unet architecture by 7.5% in dice score, 23.91% insensitivity, and 7.09% in specificity. Depth-wise separable MI-Unet showed a performance increase by 10.83% in dice score, 2.97% in sensitivity, and 12.72% in specificity as compared to the baseline Unet architecture. Hybrid Unet architecture achieved performance improvement of 9.71% in dice score, 3.56% in sensitivity, and 12.6% in specificity. Whereas the depth-wise separable hybrid Unet architecture outperformed the baseline architecture by 15.45% in dice score, 20.56% in sensitivity, and 12.22% in specificity.

Investigation of Deep Spiking Neural Networks Utilizing Gated Schottky Diode as Synaptic Devices

Deep learning produces a remarkable performance in various applications such as image classification and speech recognition. However, state-of-the-art deep neural networks require a large number of weights and enormous computation power, which results in a bottleneck of efficiency for edge-device applications. To resolve these problems, deep spiking neural networks (DSNNs) have been proposed, given the specialized synapse and neuron hardware. In this work, the hardware neuromorphic system of DSNNs with gated Schottky diodes was investigated. Gated Schottky diodes have a near-linear conductance response, which can easily implement quantized weights in synaptic devices. Based on modeling of synaptic devices, two-layer fully connected neural networks are trained by off-chip learning. The adaptation of a neuron's threshold is proposed to reduce the accuracy degradation caused by the conversion from analog neural networks (ANNs) to event-driven DSNNs. Using left-justified rate coding as an input encoding method enables low-latency classification. The effect of device variation and noisy images to the classification accuracy is investigated. The time-to-first-spike (TTFS) scheme can significantly reduce power consumption by reducing the number of firing spikes compared to a max-firing scheme.

Anatomical labeling of intracranial arteries with deep learning in patients with cerebrovascular disease

Brain arteries are routinely imaged in the clinical setting by various modalities, e.g., time-of-flight magnetic resonance angiography (TOF-MRA). These imaging techniques have great potential for the diagnosis of cerebrovascular disease, disease progression, and response to treatment. Currently, however, only qualitative assessment is implemented in clinical applications, relying on visual inspection. While manual or semi-automated approaches for quantification exist, such solutions are impractical in the clinical setting as they are time-consuming, involve too many processing steps, and/or neglect image intensity information. In this study, we present a deep learning-based solution for the anatomical labeling of intracranial arteries that utilizes complete information from 3D TOF-MRA images. We adapted and trained a state-of-the-art multi-scale Unet architecture using imaging data of 242 patients with cerebrovascular disease to distinguish 24 arterial segments. The proposed model utilizes vessel-specific information as well as raw image intensity information, and can thus take tissue characteristics into account. Our method yielded a performance of 0.89 macro F1 and 0.90 balanced class accuracy (bAcc) in labeling aggregated segments and 0.80 macro F1 and 0.83 bAcc in labeling detailed arterial segments on average. In particular, a higher F1 score than 0.75 for most arteries of clinical interest for cerebrovascular disease was achieved, with higher than 0.90 F1 scores in the larger, main arteries. Due to minimal pre-processing, simple usability, and fast predictions, our method could be highly applicable in the clinical setting.

Novel Radiomic Measurements of Tumor-Associated Vasculature Morphology on Clinical Imaging as a Biomarker of Treatment Response in Multiple Cancers

PURPOSE

The tumor-associated vasculature (TAV) differs from healthy blood vessels by its convolutedness, leakiness, and chaotic architecture, and these attributes facilitate the creation of a treatment-resistant tumor microenvironment. Measurable differences in these attributes might also help stratify patients by likely benefit of systemic therapy (e.g., chemotherapy). In this work, we present a new category of computational image-based biomarkers called quantitative tumor-associated vasculature (QuanTAV) features, and demonstrate their ability to predict response and survival across multiple cancer types, imaging modalities, and treatment regimens involving chemotherapy.

EXPERIMENTAL DESIGN

We isolated tumor vasculature and extracted mathematical measurements of twistedness and organization from routine pretreatment radiology (CT or contrast-enhanced MRI) of a total of 558 patients, who received one of four first-line chemotherapy-based therapeutic intervention strategies for breast (n = 371) or non-small cell lung cancer (NSCLC, n = 187).

RESULTS

Across four chemotherapy-based treatment strategies, classifiers of QuanTAV measurements significantly (P < 0.05) predicted response in held out testing cohorts alone (AUC = 0.63-0.71) and increased AUC by 0.06-0.12 when added to models of significant clinical variables alone. Similarly, we derived QuanTAV risk scores that were prognostic of recurrence-free survival in treatment cohorts who received surgery following chemotherapy for breast cancer [P = 0.0022; HR = 1.25; 95% confidence interval (CI), 1.08-1.44; concordance index (C-index) = 0.66] and chemoradiation for NSCLC (P = 0.039; HR = 1.28; 95% CI, 1.01-1.62; C-index = 0.66). From vessel-based risk scores, we further derived categorical QuanTAV high/low risk groups that were independently prognostic among all treatment groups, including patients with NSCLC who received chemotherapy only (P = 0.034; HR = 2.29; 95% CI, 1.07-4.94; C-index = 0.62). QuanTAV response and risk scores were independent of clinicopathologic risk factors and matched or exceeded models of clinical variables including posttreatment response.

CONCLUSIONS

Across these domains, we observed an association of vascular morphology on CT and MRI-as captured by metrics of vessel curvature, torsion, and organizational heterogeneity-and treatment outcome. Our findings suggest the potential of shape and structure of the TAV in developing prognostic and predictive biomarkers for multiple cancers and different treatment strategies.

Deep learning-based segmentation and classification of leaf images for detection of tomato plant disease

Plants contribute significantly to the global food supply. Various Plant diseases can result in production losses, which can be avoided by maintaining vigilance. However, manually monitoring plant diseases by agriculture experts and botanists is time-consuming, challenging and error-prone. To reduce the risk of disease severity, machine vision technology (i.e., artificial intelligence) can play a significant role. In the alternative method, the severity of the disease can be diminished through computer technologies and the cooperation of humans. These methods can also eliminate the disadvantages of manual observation. In this work, we proposed a solution to detect tomato plant disease using a deep leaning-based system utilizing the plant leaves image data. We utilized an architecture for deep learning based on a recently developed convolutional neural network that is trained over 18,161 segmented and non-segmented tomato leaf images-using a supervised learning approach to detect and recognize various tomato diseases using the Inception Net model in the research work. For the detection and segmentation of disease-affected regions, two state-of-the-art semantic segmentation models, i.e., U-Net and Modified U-Net, are utilized in this work. The plant leaf pixels are binary and classified by the model as Region of Interest (ROI) and background. There is also an examination of the presentation of binary arrangement (healthy and diseased leaves), six-level classification (healthy and other ailing leaf groups), and ten-level classification (healthy and other types of ailing leaves) models. The Modified U-net segmentation model outperforms the simple U-net segmentation model by 98.66 percent, 98.5 IoU score, and 98.73 percent on the dice. InceptionNet1 achieves 99.95% accuracy for binary classification problems and 99.12% for classifying six segmented class images; InceptionNet outperformed the Modified U-net model to achieve higher accuracy. The experimental results of our proposed method for classifying plant diseases demonstrate that it outperforms the methods currently available in the literature.

An analysis of the influence of transfer learning when measuring the tortuosity of blood vessels

BACKGROUND AND OBJECTIVE

Convolutional Neural Networks (CNNs) can provide excellent results regarding the segmentation of blood vessels. One important aspect of CNNs is that they can be trained on large amounts of data and then be made available, for instance, in image processing software. The pre-trained CNNs can then be easily applied in downstream blood vessel characterization tasks, such as the calculation of the length, tortuosity, or caliber of the blood vessels. Yet, it is still unclear if pre-trained CNNs can provide robust, unbiased, results in downstream tasks involving the morphological analysis of blood vessels. Here, we focus on measuring the tortuosity of blood vessels and investigate to which extent CNNs may provide biased tortuosity values even after fine-tuning the network to a new dataset under study.

METHODS

We develop a procedure for quantifying the influence of CNN pre-training in downstream analyses involving the measurement of morphological properties of blood vessels. Using the methodology, we compare the performance of CNNs that were trained on images containing blood vessels having high tortuosity with CNNs that were trained on blood vessels with low tortuosity and fine-tuned on blood vessels with high tortuosity. The opposite situation is also investigated.

RESULTS

We show that the tortuosity values obtained by a CNN trained from scratch on a dataset may not agree with those obtained by a fine-tuned network that was pre-trained on a dataset having different tortuosity statistics. In addition, we show that improving the segmentation accuracy does not necessarily lead to better tortuosity estimation. To mitigate the aforementioned issues, we propose the application of data augmentation techniques even in situations where they do not improve segmentation performance. For instance, we found that the application of elastic transformations was enough to prevent an underestimation of 8% of blood vessel tortuosity when applying CNNs to different datasets.

CONCLUSIONS

The results highlight the importance of developing new methodologies for training CNNs with the specific goal of reducing the error of morphological measurements, as opposed to the traditional approach of using segmentation accuracy as a proxy metric for performance evaluation.

DS6, Deformation-Aware Semi-Supervised Learning: Application to Small Vessel Segmentation with Noisy Training Data

Blood vessels of the brain provide the human brain with the required nutrients and oxygen. As a vulnerable part of the cerebral blood supply, pathology of small vessels can cause serious problems such as Cerebral Small Vessel Diseases (CSVD). It has also been shown that CSVD is related to neurodegeneration, such as Alzheimer’s disease. With the advancement of 7 Tesla MRI systems, higher spatial image resolution can be achieved, enabling the depiction of very small vessels in the brain. Non-Deep Learning-based approaches for vessel segmentation, e.g., Frangi’s vessel enhancement with subsequent thresholding, are capable of segmenting medium to large vessels but often fail to segment small vessels. The sensitivity of these methods to small vessels can be increased by extensive parameter tuning or by manual corrections, albeit making them time-consuming, laborious, and not feasible for larger datasets. This paper proposes a deep learning architecture to automatically segment small vessels in 7 Tesla 3D Time-of-Flight (ToF) Magnetic Resonance Angiography (MRA) data. The algorithm was trained and evaluated on a small imperfect semi-automatically segmented dataset of only 11 subjects; using six for training, two for validation, and three for testing. The deep learning model based on U-Net Multi-Scale Supervision was trained using the training subset and was made equivariant to elastic deformations in a self-supervised manner using deformation-aware learning to improve the generalisation performance. The proposed technique was evaluated quantitatively and qualitatively against the test set and achieved a Dice score of 80.44 ± 0.83. Furthermore, the result of the proposed method was compared against a selected manually segmented region (62.07 resultant Dice) and has shown a considerable improvement (18.98%) with deformation-aware learning.

A dual-stage deep convolutional neural network for automatic diagnosis of COVID-19 and pneumonia from chest CT images

In the Coronavirus disease-2019 (COVID-19) pandemic, for fast and accurate diagnosis of a large number of patients, besides traditional methods, automated diagnostic tools are now extremely required. In this paper, a deep convolutional neural network (CNN) based scheme is proposed for automated accurate diagnosis of COVID-19 from lung computed tomography (CT) scan images. First, for the automated segmentation of lung regions in a chest CT scan, a modified CNN architecture, namely SKICU-Net is proposed by incorporating additional skip interconnections in the U-Net model that overcome the loss of information in dimension scaling. Next, an agglomerative hierarchical clustering is deployed to eliminate the CT slices without significant information. Finally, for effective feature extraction and diagnosis of COVID-19 and pneumonia from the segmented lung slices, a modified DenseNet architecture, namely P-DenseCOVNet is designed where parallel convolutional paths are introduced on top of the conventional DenseNet model for getting better performance through overcoming the loss of positional arguments. Outstanding performances have been achieved with an F1 score of 0.97 in the segmentation task along with an accuracy of 87.5% in diagnosing COVID-19, common pneumonia, and normal cases. Significant experimental results and comparison with other studies show that the proposed scheme provides very satisfactory performances and can serve as an effective diagnostic tool in the current pandemic.

Automated detection and segmentation of non-small cell lung cancer computed tomography images

Detection and segmentation of abnormalities on medical images is highly important for patient management including diagnosis, radiotherapy, response evaluation, as well as for quantitative image research. We present a fully automated pipeline for the detection and volumetric segmentation of non-small cell lung cancer (NSCLC) developed and validated on 1328 thoracic CT scans from 8 institutions. Along with quantitative performance detailed by image slice thickness, tumor size, image interpretation difficulty, and tumor location, we report an in-silico prospective clinical trial, where we show that the proposed method is faster and more reproducible compared to the experts. Moreover, we demonstrate that on average, radiologists & radiation oncologists preferred automatic segmentations in 56% of the cases. Additionally, we evaluate the prognostic power of the automatic contours by applying RECIST criteria and measuring the tumor volumes. Segmentations by our method stratified patients into low and high survival groups with higher significance compared to those methods based on manual contours.

Correct interpretation of computer tomography (CT) scans is important for the correct assessment of a patient’s disease but can be subjective and timely. Here, the authors develop a system that can automatically segment the non-small cell lung cancer on CT images of patients and show in an in silico trial that the method was faster and more reproducible than clinicians.

DNL-Net: deformed non-local neural network for blood vessel segmentation

BACKGROUND

The non-local module has been primarily used in literature to capturing long-range dependencies. However, it suffers from prohibitive computational complexity and lacks the interactions among positions across the channels.

METHODS

We present a deformed non-local neural network (DNL-Net) for medical image segmentation, which has two prominent components; deformed non-local module (DNL) and multi-scale feature fusion. The former optimizes the structure of the non-local block (NL), hence, reduces the problem of excessive computation and memory usage, significantly. The latter is derived from the attention mechanisms to fuse the features of different levels and improve the ability to exchange information across channels. In addition, we introduce a residual squeeze and excitation pyramid pooling (RSEP) module that is like spatial pyramid pooling to effectively resample the features at different scales and improve the network receptive field.

RESULTS

The proposed method achieved 96.63% and 92.93% for Dice coefficient and mean intersection over union, respectively, on the intracranial blood vessel dataset. Also, DNL-Net attained 86.64%, 96.10%, and 98.37% for sensitivity, accuracy and area under receiver operation characteristic curve, respectively, on the DRIVE dataset.

CONCLUSIONS

The overall performance of DNL-Net outperforms other current state-of-the-art vessel segmentation methods, which indicates that the proposed network is more suitable for blood vessel segmentation, and is of great clinical significance.

Deep Learning Based Real-Time Semantic Segmentation of Cerebral Vessels and Cranial Nerves in Microvascular Decompression Scenes

Automatic extraction of cerebral vessels and cranial nerves has important clinical value in the treatment of trigeminal neuralgia (TGN) and hemifacial spasm (HFS). However, because of the great similarity between different cerebral vessels and between different cranial nerves, it is challenging to segment cerebral vessels and cranial nerves in real time on the basis of true-color microvascular decompression (MVD) images. In this paper, we propose a lightweight, fast semantic segmentation Microvascular Decompression Network (MVDNet) for MVD scenarios which achieves a good trade-off between segmentation accuracy and speed. Specifically, we designed a Light Asymmetric Bottleneck (LAB) module in the encoder to encode context features. A Feature Fusion Module (FFM) was introduced into the decoder to effectively combine high-level semantic features and underlying spatial details. The proposed network has no pretrained model, fewer parameters, and a fast inference speed. Specifically, MVDNet achieved 76.59% mIoU on the MVD test set, has 0.72 M parameters, and has a 137 FPS speed using a single GTX 2080Ti card.

Dual U-Net-Based Conditional Generative Adversarial Network for Blood Vessel Segmentation with Reduced Cerebral MR Training Volumes

Segmenting vessels in brain images is a critical step for many medical interventions and diagnoses of illnesses. Recent advances in artificial intelligence provide better models, achieving a human-like level of expertise in many tasks. In this paper, we present a new approach to segment Time-of-Flight Magnetic Resonance Angiography (TOF-MRA) images, relying on fewer training samples than state-of-the-art methods. We propose a conditional generative adversarial network with an adapted generator based on a concatenated U-Net with a residual U-Net architecture (UUr-cGAN) to carry out blood vessel segmentation in TOF-MRA images, relying on data augmentation to diminish the drawback of having few volumes at disposal for training the model, while preventing overfitting by using regularization techniques. The proposed model achieves 89.52% precision and 87.23% in Dice score on average from the cross-validated experiment for brain blood vessel segmentation tasks, which is similar to other state-of-the-art methods while using considerably fewer training samples. UUr-cGAN extracts important features from small datasets while preventing overfitting compared to other CNN-based methods and still achieve a relatively good performance in image segmentation tasks such as brain blood vessels from TOF-MRA.

BrainSec: Automated Brain Tissue Segmentation Pipeline for Scalable Neuropathological Analysis

As neurodegenerative disease pathological hallmarks have been reported in both grey matter (GM) and white matter (WM) with different density distributions, automating the segmentation process of GM/WM would be extremely advantageous for aiding in neuropathologic deep phenotyping. Standard segmentation methods typically involve manual annotations, where a trained researcher traces the delineation of GM/WM in ultra-high-resolution Whole Slide Images (WSIs). This method can be time-consuming and subjective, preventing a scalable analysis on pathology images. This paper proposes an automated segmentation pipeline (BrainSec) combining a Convolutional Neural Network (CNN) module for segmenting GM/WM regions and a post-processing module to remove artifacts/residues of tissues. The final output generates XML annotations that can be visualized via Aperio ImageScope. First, we investigate two baseline models for medical image segmentation: FCN, and U-Net. Then we propose a patch-based approach, BrainSec, to classify the GM/WM/background regions. We demonstrate BrainSec is robust and has reliable performance by testing it on over 180 WSIs that incorporate numerous unique cases as well as distinct neuroanatomic brain regions. We also apply gradient-weighted class activation mapping (Grad-CAM) to interpret the segmentation masks and provide relevant explanations and insights. In addition, we have integrated BrainSec with an existing Amyloid-β pathology classification model into a unified framework (without incurring significant computation complexity) to identify pathologies, visualize their distributions, and quantify each type of pathologies in segmented GM/WM regions, respectively.

Toward Sharing Brain Images: Differentially Private TOF-MRA Images With Segmentation Labels Using Generative Adversarial Networks

Sharing labeled data is crucial to acquire large datasets for various Deep Learning applications. In medical imaging, this is often not feasible due to privacy regulations. Whereas anonymization would be a solution, standard techniques have been shown to be partially reversible. Here, synthetic data using a Generative Adversarial Network (GAN) with differential privacy guarantees could be a solution to ensure the patient's privacy while maintaining the predictive properties of the data. In this study, we implemented a Wasserstein GAN (WGAN) with and without differential privacy guarantees to generate privacy-preserving labeled Time-of-Flight Magnetic Resonance Angiography (TOF-MRA) image patches for brain vessel segmentation. The synthesized image-label pairs were used to train a U-net which was evaluated in terms of the segmentation performance on real patient images from two different datasets. Additionally, the Fréchet Inception Distance (FID) was calculated between the generated images and the real images to assess their similarity. During the evaluation using the U-Net and the FID, we explored the effect of different levels of privacy which was represented by the parameter ϵ. With stricter privacy guarantees, the segmentation performance and the similarity to the real patient images in terms of FID decreased. Our best segmentation model, trained on synthetic and private data, achieved a Dice Similarity Coefficient (DSC) of 0.75 for ϵ = 7.4 compared to 0.84 for ϵ = ∞ in a brain vessel segmentation paradigm (DSC of 0.69 and 0.88 on the second test set, respectively). We identified a threshold of ϵ <5 for which the performance (DSC <0.61) became unstable and not usable. Our synthesized labeled TOF-MRA images with strict privacy guarantees retained predictive properties necessary for segmenting the brain vessels. Although further research is warranted regarding generalizability to other imaging modalities and performance improvement, our results mark an encouraging first step for privacy-preserving data sharing in medical imaging.

Medical Image-Based Computational Fluid Dynamics and Fluid-Structure Interaction Analysis in Vascular Diseases

Hemodynamic factors, induced by pulsatile blood flow, play a crucial role in vascular health and diseases, such as the initiation and progression of atherosclerosis. Computational fluid dynamics, finite element analysis, and fluid-structure interaction simulations have been widely used to quantify detailed hemodynamic forces based on vascular images commonly obtained from computed tomography angiography, magnetic resonance imaging, ultrasound, and optical coherence tomography. In this review, we focus on methods for obtaining accurate hemodynamic factors that regulate the structure and function of vascular endothelial and smooth muscle cells. We describe the multiple steps and recent advances in a typical patient-specific simulation pipeline, including medical imaging, image processing, spatial discretization to generate computational mesh, setting up boundary conditions and solver parameters, visualization and extraction of hemodynamic factors, and statistical analysis. These steps have not been standardized and thus have unavoidable uncertainties that should be thoroughly evaluated. We also discuss the recent development of combining patient-specific models with machine-learning methods to obtain hemodynamic factors faster and cheaper than conventional methods. These critical advances widen the use of biomechanical simulation tools in the research and potential personalized care of vascular diseases.