STHarDNet: Swin Transformer with HarDNet for MRI Segmentation

HUT: Hybrid UNet transformer for brain lesion and tumour segmentation

A supervised deep learning network like the UNet has performed well in segmenting brain anomalies such as lesions and tumours. However, such methods were proposed to perform on single-modality or multi-modality images. We use the Hybrid UNet Transformer (HUT) to improve performance in single-modality lesion segmentation and multi-modality brain tumour segmentation. The HUT consists of two pipelines running in parallel, one of which is UNet-based and the other is Transformer-based. The Transformer-based pipeline relies on feature maps in the intermediate layers of the UNet decoder during training. The HUT network takes in the available modalities of 3D brain volumes and embeds the brain volumes into voxel patches. The transformers in the system improve global attention and long-range correlation between the voxel patches. In addition, we introduce a self-supervised training approach in the HUT framework to enhance the overall segmentation performance. We demonstrate that HUT performs better than the state-of-the-art network SPiN in the single-modality segmentation on Anatomical Tracings of Lesions After Stroke (ATLAS) dataset by 4.84% of Dice score and a significant 41% in the Hausdorff Distance score. HUT also performed well on brain scans in the Brain Tumour Segmentation (BraTS20) dataset and demonstrated an improvement over the state-of-the-art network nnUnet by 0.96% in the Dice score and 4.1% in the Hausdorff Distance score.

Hybrid UNet transformer architecture for ischemic stoke segmentation with MRI and CT datasets

A hybrid UNet and Transformer (HUT) network is introduced to combine the merits of the UNet and Transformer architectures, improving brain lesion segmentation from MRI and CT scans. The HUT overcomes the limitations of conventional approaches by utilizing two parallel stages: one based on UNet and the other on Transformers. The Transformer-based stage captures global dependencies and long-range correlations. It uses intermediate feature vectors from the UNet decoder and improves segmentation accuracy by enhancing the attention and relationship modeling between voxel patches derived from the 3D brain volumes. In addition, HUT incorporates self-supervised learning on the transformer network. This allows the transformer network to learn by maintaining consistency between the classification layers of the different resolutions of patches and augmentations. There is an improvement in the rate of convergence of the training and the overall capability of segmentation. Experimental results on benchmark datasets, including ATLAS and ISLES2018, demonstrate HUT's advantage over the state-of-the-art methods. HUT achieves higher Dice scores and reduced Hausdorff Distance scores in single-modality and multi-modality lesion segmentation. HUT outperforms the state-the-art network SPiN in the single-modality MRI segmentation on Anatomical Tracings of lesion After Stroke (ATLAS) dataset by 4.84% of Dice score and a large margin of 40.7% in the Hausdorff Distance score. HUT also performed well on CT perfusion brain scans in the Ischemic Stroke Lesion Segmentation (ISLES2018) dataset and demonstrated an improvement over the recent state-of-the-art network USSLNet by 3.3% in the Dice score and 12.5% in the Hausdorff Distance score. With the analysis of both single and multi-modalities datasets (ATLASR12 and ISLES2018), we show that HUT can perform and generalize well on different datasets. Code is available at: https://github.com/vicsohntu/HUT_CT.

Improved STMask R-CNN-based defect detection model for automatic visual inspection of an optics lens

A lens defect is a common quality issue that has seriously harmed the scattering characteristics and performance of optical elements, reducing the quality consistency of the finished products. Furthermore, the energy hotspots coming from the high-energy laser through diffraction of optical component defects are amplified step by step in multi-level laser conduction, causing serious damage to the optical system. Traditional manual detection mainly relies on experienced workers under a special light source environment with high labor intensity, low efficiency, and accuracy. The common machine vision techniques are incapable of detecting low contrast and complex morphological defects. To address these challenges, a deep learning-based method, named STMask R-CNN, is proposed to detect defects on the surface and inside of a lens in complex environments. A Swin Transformer, which focuses on improving the modeling and representation capability of the features in order to improve the detection performance, is incorporated into the Mask R-CNN in this case. A challenge dataset containing more than 3800 images (18000 defect sample targets) with five different types of optical lens defects was created to verify the proposed approach. According to our experiments, the presented STMask R-CNN reached a precision value of 98.2%, recall value of 97.7%, F1 score of 97.9%, mAP@0.5 value of 98.1%, and FPS value of 24 f/s, which outperformed the SSD, Faster R-CNN, and YOLOv5. The experimental results demonstrated that the proposed STMask R-CNN outperformed other popular methods for multiscale targets, low contrast target detection and nesting, stacking, and intersecting defects sample detection, exhibiting good generalizability and robustness, as well as detection speed to meet mechanical equipment production efficiency requirements. In general, this research offers a favorable deep learning-based method for real-time automatic detection of optical lens defects.

An appraisal of the performance of AI tools for chronic stroke lesion segmentation

Automated demarcation of stoke lesions from monospectral magnetic resonance imaging scans is extremely useful for diverse research and clinical applications, including lesion-symptom mapping to explain deficits and predict recovery. There is a significant surge of interest in the development of supervised artificial intelligence (AI) methods for that purpose, including deep learning, with a performance comparable to trained experts. Such AI-based methods, however, require copious amounts of data. Thanks to the availability of large datasets, the development of AI-based methods for lesion segmentation has immensely accelerated in the last decade. One of these datasets is the Anatomical Tracings of Lesions After Stroke (ATLAS) dataset which includes T1-weighted images from hundreds of chronic stroke survivors with their manually traced lesions. This systematic review offers an appraisal of the impact of the ATLAS dataset in promoting the development of AI-based segmentation of stroke lesions. An examination of all published studies, that used the ATLAS dataset to both train and test their methods, highlighted an overall moderate performance (median Dice index = 59.40%) and a huge variability across studies in terms of data preprocessing, data augmentation, AI architecture, and the mode of operation (two-dimensional versus three-dimensional methods). Perhaps most importantly, almost all AI tools were borrowed from existing AI architectures in computer vision, as 90% of all selected studies relied on conventional convolutional neural network-based architectures. Overall, current research has not led to the development of robust AI architectures than can handle spatially heterogenous lesion patterns. This review also highlights the difficulty of gauging the performance of AI tools in the presence of uncertainties in the definition of the ground truth.

Hybrid Contextual Semantic Network for Accurate Segmentation and Detection of Small-Size Stroke Lesions From MRI

Stroke is a cerebrovascular disease with high mortality and disability rates. The occurrence of the stroke typically produces lesions of different sizes, with the accurate segmentation and detection of small-size stroke lesions being closely related to the prognosis of patients. However, the large lesions are usually correctly identified, the small-size lesions are usually ignored. This article provides a hybrid contextual semantic network (HCSNet) that can accurately and simultaneously segment and detect small-size stroke lesions from magnetic resonance images. HCSNet inherits the advantages of the encoder-decoder architecture and applies a novel hybrid contextual semantic module that generates high-quality contextual semantic features from the spatial and channel contextual semantic features through the skip connection layer. Moreover, a mixing-loss function is proposed to optimize HCSNet for unbalanced small-size lesions. HCSNet is trained and evaluated on 2D magnetic resonance images produced from the Anatomical Tracings of Lesions After Stroke challenge (ATLAS R2.0). Extensive experiments demonstrate that HCSNet outperforms several other state-of-the-art methods in its ability to segment and detect small-size stroke lesions. Visualization and ablation experiments reveal that the hybrid semantic module improves the segmentation and detection performance of HCSNet.

Transformer-based multilevel region and edge aggregation network for magnetic resonance image segmentation

To improve the quality of magnetic resonance (MR) image edge segmentation, some researchers applied additional edge labels to train the network to extract edge information and aggregate it with region information. They have made significant progress. However, due to the intrinsic locality of convolution operations, the convolution neural network-based region and edge aggregation has limitations in modeling long-range information. To solve this problem, we proposed a novel transformer-based multilevel region and edge aggregation network for MR image segmentation. To the best of our knowledge, this is the first literature on transformer-based region and edge aggregation. We first extract multilevel region and edge features using a dual-branch module. Then, the region and edge features at different levels are inferred and aggregated through multiple transformer-based inference modules to form multilevel complementary features. Finally, the attention feature selection module aggregates these complementary features with the corresponding level region and edge features to decode the region and edge features. We evaluated our method on a public MR dataset: Medical image computation and computer-assisted intervention atrial segmentation challenge (ASC). Meanwhile, the private MR dataset considered infrapatellar fat pad (IPFP). Our method achieved a dice score of 93.2% for ASC and 91.9% for IPFP. Compared with other 2D segmentation methods, our method improved a dice score by 0.6% for ASC and 3.0% for IPFP.

Application of Artificial Intelligence in the Practice of Medicine

Advancements in artificial intelligence (AI) based on machine and deep learning are transforming certain medical disciplines [...]