DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

Unsupervised Domain Adaptation of Object Detectors: A Survey

Recent advances in deep learning have led to the development of accurate and efficient models for various computer vision applications such as classification, segmentation, and detection. However, learning highly accurate models relies on the availability of large-scale annotated datasets. Due to this, model performance drops drastically when evaluated on label-scarce datasets having visually distinct images, termed as domain adaptation problem. There are a plethora of works to adapt classification and segmentation models to label-scarce target dataset through unsupervised domain adaptation. Considering that detection is a fundamental task in computer vision, many recent works have focused on developing novel domain adaptive detection techniques. Here, we describe in detail the domain adaptation problem for detection and present an extensive survey of the various methods. Furthermore, we highlight strategies proposed and the associated shortcomings. Subsequently, we identify multiple aspects of the problem that are most promising for future research. We believe that this survey shall be valuable to the pattern recognition experts working in the fields of computer vision, biometrics, medical imaging, and autonomous navigation by introducing them to the problem, and familiarizing them with the current status of the progress while providing promising directions for future research.

Multi-scale nested UNet with transformer for colorectal polyp segmentation

BACKGROUND

Polyp detection and localization are essential tasks for colonoscopy. U-shape network based convolutional neural networks have achieved remarkable segmentation performance for biomedical images, but lack of long-range dependencies modeling limits their receptive fields.

PURPOSE

Our goal was to develop and test a novel architecture for polyp segmentation, which takes advantage of learning local information with long-range dependencies modeling.

METHODS

A novel architecture combining with multi-scale nested UNet structure integrated transformer for polyp segmentation was developed. The proposed network takes advantage of both CNN and transformer to extract distinct feature information. The transformer layer is embedded between the encoder and decoder of a U-shape net to learn explicit global context and long-range semantic information. To address the challenging of variant polyp sizes, a MSFF unit was proposed to fuse features with multiple resolution.

RESULTS

Four public datasets and one in-house dataset were used to train and test the model performance. Ablation study was also conducted to verify each component of the model. For dataset Kvasir-SEG and CVC-ClinicDB, the proposed model achieved mean dice score of 0.942 and 0.950 respectively, which were more accurate than the other methods. To show the generalization of different methods, we processed two cross dataset validations, the proposed model achieved the highest mean dice score. The results demonstrate that the proposed network has powerful learning and generalization capability, significantly improving segmentation accuracy and outperforming state-of-the-art methods.

CONCLUSIONS

The proposed model produced more accurate polyp segmentation than current methods on four different public and one in-house datasets. Its capability of polyps segmentation in different sizes shows the potential clinical application.

FE-Net: Feature enhancement segmentation network

Semantic segmentation is one of the directions in image research. It aims to obtain the contours of objects of interest, facilitating subsequent engineering tasks such as measurement and feature selection. However, existing segmentation methods still lack precision in class edge, particularly in multi-class mixed region. To this end, we present the Feature Enhancement Network (FE-Net), a novel approach that leverages edge label and pixel-wise weights to enhance segmentation performance in complex backgrounds. Firstly, we propose a Smart Edge Head (SE-Head) to process shallow-level information from the backbone network. It is combined with the FCN-Head and SepASPP-Head, located at deeper layers, to form a transitional structure where the loss weights gradually transition from edge labels to semantic labels and a mixed loss is also designed to support this structure. Additionally, we propose a pixel-wise weight evaluation method, a pixel-wise weight block, and a feature enhancement loss to improve training effectiveness in multi-class regions. FE-Net achieves significant performance improvements over baselines on publicly datasets Pascal VOC2012, SBD, and ATR, with best mIoU enhancements of 15.19%, 1.42% and 3.51%, respectively. Furthermore, experiments conducted on Pole&Hole match dataset from our laboratory environment demonstrate the superior effectiveness of FE-Net in segmenting defined key pixels.

Neural Net-Enhanced Competitive Swarm Optimizer for Large-Scale Multiobjective Optimization

The competitive swarm optimizer (CSO) classifies swarm particles into loser and winner particles and then uses the winner particles to efficiently guide the search of the loser particles. This approach has very promising performance in solving large-scale multiobjective optimization problems (LMOPs). However, most studies of CSOs ignore the evolution of the winner particles, although their quality is very important for the final optimization performance. Aiming to fill this research gap, this article proposes a new neural net-enhanced CSO for solving LMOPs, called NN-CSO, which not only guides the loser particles via the original CSO strategy, but also applies our trained neural network (NN) model to evolve winner particles. First, the swarm particles are classified into winner and loser particles by the pairwise competition. Then, the loser particles and winner particles are, respectively, treated as the input and desired output to train the NN model, which tries to learn promising evolutionary dynamics by driving the loser particles toward the winners. Finally, when model training is complete, the winner particles are evolved by the well-trained NN model, while the loser particles are still guided by the winner particles to maintain the search pattern of CSOs. To evaluate the performance of our designed NN-CSO, several LMOPs with up to ten objectives and 1000 decision variables are adopted, and the experimental results show that our designed NN model can significantly improve the performance of CSOs and shows some advantages over several state-of-the-art large-scale multiobjective evolutionary algorithms as well as over model-based evolutionary algorithms.

From quantitative metrics to clinical success: assessing the utility of deep learning for tumor segmentation in breast surgery

PURPOSE

Preventing positive margins is essential for ensuring favorable patient outcomes following breast-conserving surgery (BCS). Deep learning has the potential to enable this by automatically contouring the tumor and guiding resection in real time. However, evaluation of such models with respect to pathology outcomes is necessary for their successful translation into clinical practice.

METHODS

Sixteen deep learning models based on established architectures in the literature are trained on 7318 ultrasound images from 33 patients. Models are ranked by an expert based on their contours generated from images in our test set. Generated contours from each model are also analyzed using recorded cautery trajectories of five navigated BCS cases to predict margin status. Predicted margins are compared with pathology reports.

RESULTS

The best-performing model using both quantitative evaluation and our visual ranking framework achieved a mean Dice score of 0.959. Quantitative metrics are positively associated with expert visual rankings. However, the predictive value of generated contours was limited with a sensitivity of 0.750 and a specificity of 0.433 when tested against pathology reports.

CONCLUSION

We present a clinical evaluation of deep learning models trained for intraoperative tumor segmentation in breast-conserving surgery. We demonstrate that automatic contouring is limited in predicting pathology margins despite achieving high performance on quantitative metrics.

Deep Upscale U-Net for automatic tongue segmentation

In a treatment or diagnosis related to oral health conditions such as oral cancer and oropharyngeal cancer, an investigation of tongue's movements is a major part. In an automatic measurement of such movement, it must first start with a task of tongue segmentation. This paper proposes a solution of tongue segmentation based on a decoder-encoder CNN-based structure i.e., U-Net. However, it could suffer from a problem of feature loss in deep layers. This paper proposes a Deep Upscale U-Net (DU-UNET). An additional up-sampling of the feature map from a contracting path is concatenated to an upper layer of an expansive path, based on an original U-Net structure. The segmentation model is constructed by training DU-UNET on the two publicly available datasets, and transferred to the self-collected dataset of tongue images with five tongue postures which were recorded at a far distance from a camera under a real-world scenario. The proposed DU-UNET outperforms the other existing methods in our literature reviews, with accuracy of 99.2%, mean IoU of 97.8%, Dice score of 96.8%, and Jaccard score of 96.8%.

Evaluation of Dental Plaque Area with Artificial Intelligence Model

OBJECTIVES

This study aims to assess the diagnostic accuracy of an artificial intelligence (AI) system employing deep learning for identifying dental plaque, utilizing a dataset comprising photographs of permanent teeth.

MATERIALS AND METHODS

In this study, photographs of 168 teeth belonging to 20 patients aged between 10 and 15 years, who met our criteria, were included. Intraoral photographs were taken of the patients in two stages, before and after the application of the plaque staining agent. To train the AI system to identify plaque on teeth with dental plaque that is not discolored, plaque and teeth were marked on photos with exposed dental plaque. One hundred forty teeth were used to construct the training group, while 28 teeth were used to create the test group. Another dentist reviewed images of teeth with dental plaque that was not discolored, and the effectiveness of AI in detecting plaque was evaluated using pertinent performance indicators. To compare the AI model and the dentist's evaluation outcomes, the mean intersection over union (IoU) values were evaluated by the Wilcoxon test.

RESULTS

The AI system showed higher performance in our study with a precision of 82% accuracy, 84% sensitivity, 83% F1 score, 87% accuracy, and 89% specificity in plaque detection. The area under the curve (AUC) value was found to be 0.922, and the IoU value was 76%. Subsequently, the dentist's plaque diagnosis performance was also evaluated. The IoU value was 0.71, and the AUC was 0.833. The AI model showed statistically significantly higher performance than the dentist (P < 0.05).

CONCLUSIONS

The AI algorithm that we developed has achieved promising results and demonstrated clinically acceptable performance in detecting dental plaque compared to a dentist.

Automatic Lenke classification of adolescent idiopathic scoliosis with deep learning

Purpose

The Lenke classification system is widely utilized as the preoperative evaluation protocol for adolescent idiopathic scoliosis (AIS). However, manual measurement is susceptible to observer-induced variability, which consequently impacts the evaluation of progression. The goal of this investigation was to develop an automated Lenke classification system utilizing innovative deep learning algorithms.

Methods

Using the database from the First Affiliated Hospital of Sun Yat-sen University, the whole spinal x-rays images were retrospectively collected. Specifically, images collection was divided into AIS and control group. The control group consisted of individuals who underwent routine health checks and did not have scoliosis. Afterwards, relative features of all images were annotated. Deep learning was implemented through the utilization of the key-point based detection method to realize the vertebral detection, and Cobb angle measurement and scoliosis classification were performed based on relevant standards. Besides, the segmentation method was employed to achieve the recognition of lumbar vertebral pedicle to determine the type of lumbar spine modifier. Finally, the model performance was further quantitatively analyzed.

Results

In the study, a total of 2082 spinal x-ray images were collected from 407 AIS patients and 227 individuals in the control group. The model for vertebral detection achieved an F1-score of 0.809 for curve type evaluation and an F1-score of 0.901 for thoracic sagittal profile. The intraclass correlation efficient (ICC) of the Cobb angle measurement was 0.925. In the analysis of performance for vertebra pedicle segmentation model, the F1-score of lumbar modification profile was 0.942, the intersection over union (IOU) of the target pixels was 0.827, and the Hausdorff distance (HD) was 6.565 ± 2.583 mm. Specifically, the F1-score for ultimate Lenke type classifier was 0.885.

Conclusions

This study has constructed an automated Lenke classification system by employing the deep learning networks to achieve the recognition pattern and feature extraction. Our models require further validation in additional cases in the future.

The multimodality cell segmentation challenge: toward universal solutions

Cell segmentation is a critical step for quantitative single-cell analysis in microscopy images. Existing cell segmentation methods are often tailored to specific modalities or require manual interventions to specify hyper-parameters in different experimental settings. Here, we present a multimodality cell segmentation benchmark, comprising more than 1,500 labeled images derived from more than 50 diverse biological experiments. The top participants developed a Transformer-based deep-learning algorithm that not only exceeds existing methods but can also be applied to diverse microscopy images across imaging platforms and tissue types without manual parameter adjustments. This benchmark and the improved algorithm offer promising avenues for more accurate and versatile cell analysis in microscopy imaging.

Brain tumor detection with integrating traditional and computational intelligence approaches across diverse imaging modalities - Challenges and future directions

Brain tumor segmentation and classification play a crucial role in the diagnosis and treatment planning of brain tumors. Accurate and efficient methods for identifying tumor regions and classifying different tumor types are essential for guiding medical interventions. This study comprehensively reviews brain tumor segmentation and classification techniques, exploring various approaches based on image processing, machine learning, and deep learning. Furthermore, our study aims to review existing methodologies, discuss their advantages and limitations, and highlight recent advancements in this field. The impact of existing segmentation and classification techniques for automated brain tumor detection is also critically examined using various open-source datasets of Magnetic Resonance Images (MRI) of different modalities. Moreover, our proposed study highlights the challenges related to segmentation and classification techniques and datasets having various MRI modalities to enable researchers to develop innovative and robust solutions for automated brain tumor detection. The results of this study contribute to the development of automated and robust solutions for analyzing brain tumors, ultimately aiding medical professionals in making informed decisions and providing better patient care.

Diversity-Learning Block: Conquer Feature Homogenization of Multibranch

Visual Geometry Group (VGG)-style ConvNet is an neural-network process units (NPU)-friendly network; however, the accuracy of this architecture cannot keep up with other well-designed network structures. Although some reparameterization methods are proposed to remedy this weakness, their performance suffers from the homogenization issue of parallel branches, and the preset shape of convolution kernels also influences spatial perception. To address this problem, we propose a diversity-learning (DL) block to build the DLNet, which could adaptively learn various features to enrich the feature space. To balance floating point of operations (FLOPs) and accuracy, groupwise operation is introduced and finally, a lightweight DL ConvNet DLGNet is obtained. Extensive evaluations have been conducted on different computer vision tasks, e.g., image classification [Canadian Institute For Advanced Research (CIFAR) and ImageNet], object detection [PASCAL visual object classes (VOC) and Microsoft Common Objects in Context (MS COCO)], and semantic segmentation (Cityscapes). The experimental results show that our proposed DLGNet can achieve comparable performance with the state-of-the-art networks while the speed is 183% faster than GhostNet and even over 600% faster than MobileNetV3 with similar accuracy when running on NPU.

An Adaptive Image Segmentation Network for Surface Defect Detection

Surface defect detection plays an essential role in industry, and it is challenging due to the following problems: 1) the similarity between defect and nondefect texture is very high, which eventually leads to recognition or classification errors and 2) the size of defects is tiny, which are much more difficult to be detected than larger ones. To address such problems, this article proposes an adaptive image segmentation network (AIS-Net) for pixelwise segmentation of surface defects. It consists of three main parts: multishuffle-block dilated convolution (MSDC), dual attention context guidance (DACG), and adaptive category prediction (ACP) modules, where MSDC is designed to merge the multiscale defect features for avoiding the loss of tiny defect feature caused by model depth, DACG is designed to capture more contextual information from the defect feature map for locating defect regions and obtaining clear segmentation boundaries, and ACP is used to make classification and regression for predicting defect categories. Experimental results show that the proposed AIS-Net is superior to the state-of-the-art approaches on four actual surface defect datasets (NEU-DET: 98.38% ± 0.03%, DAGM: 99.25% ± 0.02%, Magnetic-tile: 98.73% ± 0.13%, and MVTec: 99.72% ± 0.02%).

Automatic assessment of DWI-ASPECTS for acute ischemic stroke based on deep learning

BACKGROUND

Alberta Stroke Program Early Computed Tomography Score (ASPECTS) is a standardized semi-quantitative method for early ischemic changes in acute ischemic stroke.

PURPOSE

However, ASPECTS is still affected by expert experience and inconsistent results between readers in clinical. This study aims to propose an automatic ASPECTS scoring model based on diffusion-weighted imaging (DWI) mode to help clinicians make accurate treatment plans.

METHODS

Eighty-two patients with stroke were included in the study. First, we designed a new deep learning network for segmenting ASPECTS scoring brain regions. The network is improved based on U-net, which integrates multiple modules. Second, we proposed using hybrid classifiers to classify brain regions. For brain regions with larger areas, we used brain grayscale comparison algorithm to train machine learning classifiers, while using hybrid feature training for brain regions with smaller areas.

RESULTS

The average DICE coefficient of the segmented hindbrain area can reach 0.864. With the proposed hybrid classifier, our method performs significantly on both region-level ASPECTS and dichotomous ASPECTS. The sensitivity and accuracy on the test set are 95.51% and 93.43%, respectively. For dichotomous ASPECTS, the intraclass correlation coefficient (ICC) between our automated ASPECTS score and the expert reading was 0.87.

CONCLUSIONS

This study proposed an automated model for ASPECTS scoring of patients with acute ischemic stroke based on DWI images. Experimental results show that the method of segmentation first and then classification is feasible. Our method has the potential to assist physicians in the Alberta Stroke Program with early CT scoring and clinical stroke diagnosis.

A Review of Nuclei Detection and Segmentation on Microscopy Images Using Deep Learning With Applications to Unbiased Stereology Counting

The detection and segmentation of stained cells and nuclei are essential prerequisites for subsequent quantitative research for many diseases. Recently, deep learning has shown strong performance in many computer vision problems, including solutions for medical image analysis. Furthermore, accurate stereological quantification of microscopic structures in stained tissue sections plays a critical role in understanding human diseases and developing safe and effective treatments. In this article, we review the most recent deep learning approaches for cell (nuclei) detection and segmentation in cancer and Alzheimer's disease with an emphasis on deep learning approaches combined with unbiased stereology. Major challenges include accurate and reproducible cell detection and segmentation of microscopic images from stained sections. Finally, we discuss potential improvements and future trends in deep learning applied to cell detection and segmentation.

INSTRAS: INfrared Spectroscopic imaging-based TRAnsformers for medical image Segmentation

Infrared (IR) spectroscopic imaging is of potentially wide use in medical imaging applications due to its ability to capture both chemical and spatial information. This complexity of the data both necessitates using machine intelligence as well as presents an opportunity to harness a high-dimensionality data set that offers far more information than today's manually-interpreted images. While convolutional neural networks (CNNs), including the well-known U-Net model, have demonstrated impressive performance in image segmentation, the inherent locality of convolution limits the effectiveness of these models for encoding IR data, resulting in suboptimal performance. In this work, we propose an INfrared Spectroscopic imaging-based TRAnsformers for medical image Segmentation (INSTRAS). This novel model leverages the strength of the transformer encoders to segment IR breast images effectively. Incorporating skip-connection and transformer encoders, INSTRAS overcomes the issue of pure convolution models, such as the difficulty of capturing long-range dependencies. To evaluate the performance of our model and existing convolutional models, we conducted training on various encoder-decoder models using a breast dataset of IR images. INSTRAS, utilizing 9 spectral bands for segmentation, achieved a remarkable AUC score of 0.9788, underscoring its superior capabilities compared to purely convolutional models. These experimental results attest to INSTRAS's advanced and improved segmentation abilities for IR imaging.

DPIF-Net: a dual path network for rural road extraction based on the fusion of global and local information

Background

Automatic extraction of roads from remote sensing images can facilitate many practical applications. However, thus far, thousands of kilometers or more of roads worldwide have not been recorded, especially low-grade roads in rural areas. Moreover, rural roads have different shapes and are influenced by complex environments and other interference factors, which has led to a scarcity of dedicated low level category road datasets.

Methods

To address these issues, based on convolutional neural networks (CNNs) and tranformers, this article proposes the Dual Path Information Fusion Network (DPIF-Net). In addition, given the severe lack of low-grade road datasets, we constructed the GaoFen-2 (GF-2) rural road dataset to address this challenge, which spans three regions in China and covers an area of over 2,300 km, almost entirely composed of low-grade roads. To comprehensively test the low-grade road extraction performance and generalization ability of the model, comparative experiments are carried out on the DeepGlobe, and Massachusetts regular road datasets.

Results

The results show that DPIF-Net achieves the highest IoU and F1 score on three datasets compared with methods such as U-Net, SegNet, DeepLabv3+, and D-LinkNet, with notable performance on the GF-2 dataset, reaching 0.6104 and 0.7608, respectively. Furthermore, multiple validation experiments demonstrate that DPIF-Net effectively preserves improved connectivity in low-grade road extraction with a modest parameter count of 63.9 MB. The constructed low-grade road dataset and proposed methods will facilitate further research on rural roads, which holds promise for assisting governmental authorities in making informed decisions and strategies to enhance rural road infrastructure.

Improved U-net network asphalt pavement crack detection method

Road crack detection is one of the important parts of road safety detection. Aiming at the problems such as weak segmentation effect of basic U-Net on pavement crack, insufficient precision of crack contour segmentation, difficult to identify narrow crack and low segmentation accuracy, this paper proposes an improved U-net network pavement crack segmentation method. VGG16 and Up_Conv (Upsampling Convolution) modules are introduced as backbone network and feature enhancement network respectively, and the more abstract features in the image are extracted by using the Block depth separable convolution blocks, and the multi-scale features are captured and enhanced by higher level semantic information to improve the recognition accuracy of narrow cracks in the road surface. The improved network embedded the Ca(Channel Attention) attention mechanism in U-net's jump connection to enhance the crack portion to suppress background noise. At the same time, DG_Conv(Depthwise GSConv Convolution) module and UnetUp(Unet Upsampling) module are added in the decoding part to extract richer features through more convolutional layers in the network, so that the model pays more attention to the detailed part of the crack, so the segmentation accuracy can be improved. In order to verify the model's ability to detect cracks in complex backgrounds, experiments were carried out on CFD and Deepcrack datasets. The experimental results show that compared with the traditional U-net network F1-score and mIoU have increased by 13.6% and 9.9% respectively. Superior to advanced models such as U-net, Segnet and Linknet in accuracy and generalization ability, the improved model provides a new method for asphalt pavement crack detection. The model is more conducive to practical application and ground deployment, and can be applied in road maintenance projects.

Transformers for Remote Sensing: A Systematic Review and Analysis

Research on transformers in remote sensing (RS), which started to increase after 2021, is facing the problem of a relative lack of review. To understand the trends of transformers in RS, we undertook a quantitative analysis of the major research on transformers over the past two years by dividing the application of transformers into eight domains: land use/land cover (LULC) classification, segmentation, fusion, change detection, object detection, object recognition, registration, and others. Quantitative results show that transformers achieve a higher accuracy in LULC classification and fusion, with more stable performance in segmentation and object detection. Combining the analysis results on LULC classification and segmentation, we have found that transformers need more parameters than convolutional neural networks (CNNs). Additionally, further research is also needed regarding inference speed to improve transformers' performance. It was determined that the most common application scenes for transformers in our database are urban, farmland, and water bodies. We also found that transformers are employed in the natural sciences such as agriculture and environmental protection rather than the humanities or economics. Finally, this work summarizes the analysis results of transformers in remote sensing obtained during the research process and provides a perspective on future directions of development.

An accurate semantic segmentation model for bean seedlings and weeds identification based on improved ERFnet

In agricultural production activities, the growth of crops always accompanies the competition of weeds for nutrients and sunlight. In order to mitigate the adverse effects of weeds on yield, we apply semantic segmentation techniques to differentiate between seedlings and weeds, leading to precision weeding. The proposed EPAnet employs a loss function coupled with Cross-entropy loss and Dice loss to enhance attention to feature information. A multi-Decoder cooperative module based on ERFnet is designed to enhance information transfer during feature mapping. The SimAM is introduced to enhance position recognition. DO-CONV is used to replace the traditional convolution Feature Pyramid Networks (FPN) connection layer to integrate feature information, improving the model's performance on leaf edge processing, and is named FDPN. Moreover, the Overall Accuracy has been improved by 0.65%, the mean Intersection over Union (mIoU) by 1.91%, and the Frequency-Weighted Intersection over Union (FWIoU) by 1.19%. Compared to other advanced methods, EPAnet demonstrates superior image segmentation results in complex natural environments with uneven lighting, leaf interference, and shadows.

Segmentation of void defects in X-ray images of chip solder joints based on PCB-DeepLabV3 algorithm

Defects within chip solder joints are usually inspected visually for defects using X-ray imaging to obtain images. The phenomenon of voids inside solder joints is one of the most likely types of defects in the soldering process, and accurate detection of voids becomes difficult due to their irregular shapes, varying sizes, and defocused edges. To address this problem, an X-ray void image segmentation algorithm based on improved PCB-DeepLabV3 is proposed. Firstly, to meet the demand for lightweight and easy deployment in industrial scenarios, mobilenetv2 is used as the feature extraction backbone network of the PCB-DeepLabV3 model; then, Attentional multi-scale two-space pyramid pooling network (AMTPNet) is designed to optimize the shallow feature edges and to improve the ability to capture detailed information; finally, image cropping and cleaning methods are designed to enhance the training dataset, and the improved PCB-DeepLabV3 is applied to the training dataset. The improved PCB-DeepLabV3 model is used to segment the void regions within the solder joints and compared with the classical semantic segmentation models such as Unet, SegNet, PSPNet, and DeeplabV3. The proposed new method enables the solder joint void inspection to get rid of the traditional way of visual inspection, realize intelligent upgrading, and effectively improve the problem of difficult segmentation of the target virtual edges, to obtain the inspection results with higher accuracy.

Automated thorax disease diagnosis using multi-branch residual attention network

Chest X-ray (CXR) is an extensively utilized radiological modality for supporting the diagnosis of chest diseases. However, existing research approaches suffer from limitations in effectively integrating multi-scale CXR image features and are also hindered by imbalanced datasets. Therefore, there is a pressing need for further advancement in computer-aided diagnosis (CAD) of thoracic diseases. To tackle these challenges, we propose a multi-branch residual attention network (MBRANet) for thoracic disease diagnosis. MBRANet comprises three components. Firstly, to address the issue of inadequate extraction of spatial and positional information by the convolutional layer, a novel residual structure incorporating a coordinate attention (CA) module is proposed to extract features at multiple scales. Next, based on the concept of a Feature Pyramid Network (FPN), we perform multi-scale feature fusion in the following manner. Thirdly, we propose a novel Multi-Branch Feature Classifier (MFC) approach, which leverages the class-specific residual attention (CSRA) module for classification instead of relying solely on the fully connected layer. In addition, the designed BCEWithLabelSmoothing loss function improves the generalization ability and mitigates the problem of class imbalance by introducing a smoothing factor. We evaluated MBRANet on the ChestX-Ray14, CheXpert, MIMIC-CXR, and IU X-Ray datasets and achieved average AUCs of 0.841, 0.895, 0.805, and 0.745, respectively. Our method outperformed state-of-the-art baselines on these benchmark datasets.

Polyp segmentation based on implicit edge-guided cross-layer fusion networks

Polyps are abnormal tissue clumps growing primarily on the inner linings of the gastrointestinal tract. While such clumps are generally harmless, they can potentially evolve into pathological tumors, and thus require long-term observation and monitoring. Polyp segmentation in gastrointestinal endoscopy images is an important stage for polyp monitoring and subsequent treatment. However, this segmentation task faces multiple challenges: the low contrast of the polyp boundaries, the varied polyp appearance, and the co-occurrence of multiple polyps. So, in this paper, an implicit edge-guided cross-layer fusion network (IECFNet) is proposed for polyp segmentation. The codec pair is used to generate an initial saliency map, the implicit edge-enhanced context attention module aggregates the feature graph output from the encoding and decoding to generate the rough prediction, and the multi-scale feature reasoning module is used to generate final predictions. Polyp segmentation experiments have been conducted on five popular polyp image datasets (Kvasir, CVC-ClinicDB, ETIS, CVC-ColonDB, and CVC-300), and the experimental results show that the proposed method significantly outperforms a conventional method, especially with an accuracy margin of 7.9% on the ETIS dataset.

Pre-operative lung ablation prediction using deep learning

OBJECTIVE

Microwave lung ablation (MWA) is a minimally invasive and inexpensive alternative cancer treatment for patients who are not candidates for surgery/radiotherapy. However, a major challenge for MWA is its relatively high tumor recurrence rates, due to incomplete treatment as a result of inaccurate planning. We introduce a patient-specific, deep-learning model to accurately predict post-treatment ablation zones to aid planning and enable effective treatments.

MATERIALS AND METHODS

Our IRB-approved retrospective study consisted of ablations with a single applicator/burn/vendor between 01/2015 and 01/2019. The input data included pre-procedure computerized tomography (CT), ablation power/time, and applicator position. The ground truth ablation zone was segmented from follow-up CT post-treatment. Novel deformable image registration optimized for ablation scans and an applicator-centric co-ordinate system for data analysis were applied. Our prediction model was based on the U-net architecture. The registrations were evaluated using target registration error (TRE) and predictions using Bland-Altman plots, Dice co-efficient, precision, and recall, compared against the applicator vendor's estimates.

RESULTS

The data included 113 unique ablations from 72 patients (median age 57, interquartile range (IQR) (49-67); 41 women). We obtained a TRE ≤ 2 mm on 52 ablations. Our prediction had no bias from ground truth ablation volumes (p = 0.169) unlike the vendor's estimate (p < 0.001) and had smaller limits of agreement (p < 0.001). An 11% improvement was achieved in the Dice score. The ability to account for patient-specific in-vivo anatomical effects due to vessels, chest wall, heart, lung boundaries, and fissures was shown.

CONCLUSIONS

We demonstrated a patient-specific deep-learning model to predict the ablation treatment effect prior to the procedure, with the potential for improved planning, achieving complete treatments, and reduce tumor recurrence.

CLINICAL RELEVANCE STATEMENT

Our method addresses the current lack of reliable tools to estimate ablation extents, required for ensuring successful ablation treatments. The potential clinical implications include improved treatment planning, ensuring complete treatments, and reducing tumor recurrence.

Automated segmentation for early detection of uveal melanoma

OBJECTIVE

Uveal melanoma is the most common intraocular malignancy in adults. Current screening and triaging methods for melanocytic choroidal tumours face inherent limitations, particularly in regions with limited access to specialized ocular oncologists. This study explores the potential of machine learning to automate tumour segmentation. We develop and evaluate a machine-learning model for lesion segmentation using ultra-wide-field fundus photography.

METHOD

A retrospective chart review was conducted of patients diagnosed with uveal melanoma, choroidal nevi, or congenital hypertrophy of the retinal pigmented epithelium at a tertiary academic medical centre. Included patients had a single ultra-wide-field fundus photograph (Optos PLC, Dunfermline, Fife, Scotland) of adequate quality to visualize the lesion of interest, as confirmed by a single ocular oncologist. These images were used to develop and test a machine-learning algorithm for lesion segmentation.

RESULTS

A total of 396 images were used to develop a machine-learning algorithm for lesion segmentation. Ninety additional images were used in the testing data set along with images of 30 healthy control individuals. Of the images with successfully detected lesions, the machine-learning segmentation yielded Dice coefficients of 0.86, 0.81, and 0.85 for uveal melanoma, choroidal nevi, and congenital hypertrophy of the retinal pigmented epithelium, respectively. Sensitivities for any lesion detection per image were 1.00, 0.90, and 0.87, respectively. For images without lesions, specificity was 0.93.

CONCLUSION

Our study demonstrates a novel machine-learning algorithm's performance, suggesting its potential clinical utility as a widely accessible method of screening choroidal tumours. Additional evaluation methods are necessary to further enhance the model's lesion classification and diagnostic accuracy.

ISLS: An Illumination-Aware Sauce-Packet Leakage Segmentation Method

The segmentation of abnormal regions is vital in smart manufacturing. The blurring sauce-packet leakage segmentation task (BSLST) is designed to distinguish the sauce packet and the leakage's foreground and background at the pixel level. However, the existing segmentation system for detecting sauce-packet leakage on intelligent sensors encounters an issue of imaging blurring caused by uneven illumination. This issue adversely affects segmentation performance, thereby hindering the measurements of leakage area and impeding the automated sauce-packet production. To alleviate this issue, we propose the two-stage illumination-aware sauce-packet leakage segmentation (ISLS) method for intelligent sensors. The ISLS comprises two main stages: illumination-aware region enhancement and leakage region segmentation. In the first stage, YOLO-Fastestv2 is employed to capture the Region of Interest (ROI), which reduces redundancy computations. Additionally, we propose image enhancement to relieve the impact of uneven illumination, enhancing the texture details of the ROI. In the second stage, we propose a novel feature extraction network. Specifically, we propose the multi-scale feature fusion module (MFFM) and the Sequential Self-Attention Mechanism (SSAM) to capture discriminative representations of leakage. The multi-level features are fused by the MFFM with a small number of parameters, which capture leakage semantics at different scales. The SSAM realizes the enhancement of valid features and the suppression of invalid features by the adaptive weighting of spatial and channel dimensions. Furthermore, we generate a self-built dataset of sauce packets, including 606 images with various leakage areas. Comprehensive experiments demonstrate that our ISLS method shows better results than several state-of-the-art methods, with additional performance analyses deployed on intelligent sensors to affirm the effectiveness of our proposed method.

BiU-net: A dual-branch structure based on two-stage fusion strategy for biomedical image segmentation

BACKGROUND AND OBJECTIVE

Computer-based biomedical image segmentation plays a crucial role in planning of assisted diagnostics and therapy. However, due to the variable size and irregular shape of the segmentation target, it is still a challenge to construct an effective medical image segmentation structure. Recently, hybrid architectures based on convolutional neural networks (CNNs) and transformers were proposed. However, most current backbones directly replace one or all convolutional layers with transformer blocks, regardless of the semantic gap between features. Thus, how to sufficiently and effectively eliminate the semantic gap as well as combine the global and local information is a critical challenge.

METHODS

To address the challenge, we propose a novel structure, called BiU-Net, which integrates CNNs and transformers with a two-stage fusion strategy. In the first fusion stage, called Single-Scale Fusion (SSF) stage, the encoding layers of the CNNs and transformers are coupled, with both having the same feature map size. The SSF stage aims to reconstruct local features based on CNNs and long-range information based on transformers in each encoding block. In the second stage, Multi-Scale Fusion (MSF), BiU-Net interacts with multi-scale features from various encoding layers to eliminate the semantic gap between deep and shallow layers. Furthermore, a Context-Aware Block (CAB) is embedded in the bottleneck to reinforce multi-scale features in the decoder.

RESULTS

Experiments on four public datasets were conducted. On the BUSI dataset, our BiU-Net achieved 85.50 % on Dice coefficient (Dice), 76.73 % on intersection over union (IoU), and 97.23 % on accuracy (ACC). Compared to the state-of-the-art method, BiU-Net improves Dice by 1.17 %. For the Monuseg dataset, the proposed method attained the highest scores, reaching 80.27 % and 67.22 % for Dice and IoU. The BiU-Net achieves 95.33 % and 81.22 % Dice on the PH2 and DRIVE datasets.

CONCLUSIONS

The results of our experiments showed that BiU-Net transcends existing state-of-the-art methods on four publicly available biomedical datasets. Due to the powerful multi-scale feature extraction ability, our proposed BiU-Net is a versatile medical image segmentation framework for various types of medical images. The source code is released on (https://github.com/ZYLandy/BiU-Net).

Robust detection and refinement of saliency identification

Salient object detection is an increasingly popular topic in the computer vision field, particularly for images with complex backgrounds and diverse object parts. Background information is an essential factor in detecting salient objects. This paper suggests a robust and effective methodology for salient object detection. This method involves two main stages. The first stage is to produce a saliency detection map based on the dense and sparse reconstruction of image regions using a refined background dictionary. The refined background dictionary uses a boundary conductivity measurement to exclude salient object regions near the image's boundary from a background dictionary. In the second stage, the CascadePSP network is integrated to refine and correct the local boundaries of the saliency mask to highlight saliency objects more uniformly. Using six evaluation indexes, experimental outcomes conducted on three datasets show that the proposed approach performs effectively compared to the state-of-the-art methods in salient object detection, particularly in identifying the challenging salient objects located near the image's boundary. These results demonstrate the potential of the proposed framework for various computer vision applications.

SLMFNet: Enhancing land cover classification of remote sensing images through selective attentions and multi-level feature fusion

Land cover classification (LCC) is of paramount importance for assessing environmental changes in remote sensing images (RSIs) as it involves assigning categorical labels to ground objects. The growing availability of multi-source RSIs presents an opportunity for intelligent LCC through semantic segmentation, offering a comprehensive understanding of ground objects. Nonetheless, the heterogeneous appearances of terrains and objects contribute to significant intra-class variance and inter-class similarity at various scales, adding complexity to this task. In response, we introduce SLMFNet, an innovative encoder-decoder segmentation network that adeptly addresses this challenge. To mitigate the sparse and imbalanced distribution of RSIs, we incorporate selective attention modules (SAMs) aimed at enhancing the distinguishability of learned representations by integrating contextual affinities within spatial and channel domains through a compact number of matrix operations. Precisely, the selective position attention module (SPAM) employs spatial pyramid pooling (SPP) to resample feature anchors and compute contextual affinities. In tandem, the selective channel attention module (SCAM) concentrates on capturing channel-wise affinity. Initially, feature maps are aggregated into fewer channels, followed by the generation of pairwise channel attention maps between the aggregated channels and all channels. To harness fine-grained details across multiple scales, we introduce a multi-level feature fusion decoder with data-dependent upsampling (MLFD) to meticulously recover and merge feature maps at diverse scales using a trainable projection matrix. Empirical results on the ISPRS Potsdam and DeepGlobe datasets underscore the superior performance of SLMFNet compared to various state-of-the-art methods. Ablation studies affirm the efficacy and precision of SAMs in the proposed model.

Vehicle Detection Algorithms for Autonomous Driving: A Review

Autonomous driving, as a pivotal technology in modern transportation, is progressively transforming the modalities of human mobility. In this domain, vehicle detection is a significant research direction that involves the intersection of multiple disciplines, including sensor technology and computer vision. In recent years, many excellent vehicle detection methods have been reported, but few studies have focused on summarizing and analyzing these algorithms. This work provides a comprehensive review of existing vehicle detection algorithms and discusses their practical applications in the field of autonomous driving. First, we provide a brief description of the tasks, evaluation metrics, and datasets for vehicle detection. Second, more than 200 classical and latest vehicle detection algorithms are summarized in detail, including those based on machine vision, LiDAR, millimeter-wave radar, and sensor fusion. Finally, this article discusses the strengths and limitations of different algorithms and sensors, and proposes future trends.

WITHDRAWN: Coronary artery segmentation based on ACMA-Net and unscented Kalman filter algorithm

This article has been withdrawn at the request of the author(s) and/or editor. The Publisher apologizes for any inconvenience this may cause. The full Elsevier Policy on Article Withdrawal can be found at https://www.elsevier.com/about/policies/article-withdrawal.

An Update on the Use of Artificial Intelligence in Digital Pathology for Oral Epithelial Dysplasia Research

INTRODUCTION

Oral epithelial dysplasia (OED) is a precancerous histopathological finding which is considered the most important prognostic indicator for determining the risk of malignant transformation into oral squamous cell carcinoma (OSCC). The gold standard for diagnosis and grading of OED is through histopathological examination, which is subject to inter- and intra-observer variability, impacting accurate diagnosis and prognosis. The aim of this review article is to examine the current advances in digital pathology for artificial intelligence (AI) applications used for OED diagnosis.

MATERIALS AND METHODS

We included studies that used AI for diagnosis, grading, or prognosis of OED on histopathology images or intraoral clinical images. Studies utilizing imaging modalities other than routine light microscopy (e.g., scanning electron microscopy), or immunohistochemistry-stained histology slides, or immunofluorescence were excluded from the study. Studies not focusing on oral dysplasia grading and diagnosis, e.g., to discriminate OSCC from normal epithelial tissue were also excluded.

RESULTS

A total of 24 studies were included in this review. Nineteen studies utilized deep learning (DL) convolutional neural networks for histopathological OED analysis, and 4 used machine learning (ML) models. Studies were summarized by AI method, main study outcomes, predictive value for malignant transformation, strengths, and limitations.

CONCLUSION

ML/DL studies for OED grading and prediction of malignant transformation are emerging as promising adjunctive tools in the field of digital pathology. These adjunctive objective tools can ultimately aid the pathologist in more accurate diagnosis and prognosis prediction. However, further supportive studies that focus on generalization, explainable decisions, and prognosis prediction are needed.

Deep learning for automatic segmentation of vestibular schwannoma: a retrospective study from multi-center routine MRI

Automatic segmentation of vestibular schwannoma (VS) from routine clinical MRI has potential to improve clinical workflow, facilitate treatment decisions, and assist patient management. Previous work demonstrated reliable automatic segmentation performance on datasets of standardized MRI images acquired for stereotactic surgery planning. However, diagnostic clinical datasets are generally more diverse and pose a larger challenge to automatic segmentation algorithms, especially when post-operative images are included. In this work, we show for the first time that automatic segmentation of VS on routine MRI datasets is also possible with high accuracy. We acquired and publicly release a curated multi-center routine clinical (MC-RC) dataset of 160 patients with a single sporadic VS. For each patient up to three longitudinal MRI exams with contrast-enhanced T1-weighted (ceT1w) (n = 124) and T2-weighted (T2w) (n = 363) images were included and the VS manually annotated. Segmentations were produced and verified in an iterative process: (1) initial segmentations by a specialized company; (2) review by one of three trained radiologists; and (3) validation by an expert team. Inter- and intra-observer reliability experiments were performed on a subset of the dataset. A state-of-the-art deep learning framework was used to train segmentation models for VS. Model performance was evaluated on a MC-RC hold-out testing set, another public VS datasets, and a partially public dataset. The generalizability and robustness of the VS deep learning segmentation models increased significantly when trained on the MC-RC dataset. Dice similarity coefficients (DSC) achieved by our model are comparable to those achieved by trained radiologists in the inter-observer experiment. On the MC-RC testing set, median DSCs were 86.2(9.5) for ceT1w, 89.4(7.0) for T2w, and 86.4(8.6) for combined ceT1w+T2w input images. On another public dataset acquired for Gamma Knife stereotactic radiosurgery our model achieved median DSCs of 95.3(2.9), 92.8(3.8), and 95.5(3.3), respectively. In contrast, models trained on the Gamma Knife dataset did not generalize well as illustrated by significant underperformance on the MC-RC routine MRI dataset, highlighting the importance of data variability in the development of robust VS segmentation models. The MC-RC dataset and all trained deep learning models were made available online.

Adaptive temporal compression for reduction of computational complexity in human behavior recognition

The research on video analytics especially in the area of human behavior recognition has become increasingly popular recently. It is widely applied in virtual reality, video surveillance, and video retrieval. With the advancement of deep learning algorithms and computer hardware, the conventional two-dimensional convolution technique for training video models has been replaced by three-dimensional convolution, which enables the extraction of spatio-temporal features. Specifically, the use of 3D convolution in human behavior recognition has been the subject of growing interest. However, the increased dimensionality has led to challenges such as the dramatic increase in the number of parameters, increased time complexity, and a strong dependence on GPUs for effective spatio-temporal feature extraction. The training speed can be considerably slow without the support of powerful GPU hardware. To address these issues, this study proposes an Adaptive Time Compression (ATC) module. Functioning as an independent component, ATC can be seamlessly integrated into existing architectures and achieves data compression by eliminating redundant frames within video data. The ATC module effectively reduces GPU computing load and time complexity with negligible loss of accuracy, thereby facilitating real-time human behavior recognition.

SC-Net: Symmetrical conical network for colorectal pathology image segmentation

BACKGROUND AND OBJECTIVE

Image segmentation of histopathology of colorectal cancer is a core task of computer aided medical image diagnosis system. Existing convolutional neural networks generally extract multi-scale information in linear flow structures by inserting multi-branch modules, which is difficult to extract heterogeneous semantic information under multi-level and different receptive field and tough to establish context dependency among different receptive field features.

METHODS

To address these issues, we propose a symmetric spiral progressive feature fusion encoder-decoder network called the Symmetric Conical Network (SC-Net). First, we design a Multi-scale Feature Extraction Block (MFEB) matching with the Symmetric Conical Network to obtain multi-branch heterogeneous semantic information under different receptive fields, so as to enrich the diversity of extracted feature information. The encoder is composed of MFEB through spiral and multi-branch arrangement to enhance context dependence between different information flow. Secondly, the information loss of contour, color and others in high-level semantic information through causally stacking MFEB, the Feature Mapping Layer (FML) is designed to map low-level features to high-level semantic features along the down-sampling branch and solve the problem of insufficient global feature extraction in deep levels.

RESULTS

The SC-Net was evaluated on our self-constructed colorectal cancer dataset, a publicly available breast cancer dataset and a polyp dataset. The results revealed that the mDice of segmentation reached 0.8611, 0.7259 and 0.7144. We compare our model with the state-of-art semantic segmentation UNet++, PSPNet, Attention U-Net, R2U-Net and other advanced segmentation networks. The experimental results demonstrate that we achieve the most advanced performance.

CONCLUSIONS

The results indicate that the proposed SC-Net excels in segmenting H&E stained pathology images, effectively preserving morphological features and spatial information even in scenarios with weak texture, poor contrast, and variations in appearance.

YOLOv8-MU: An Improved YOLOv8 Underwater Detector Based on a Large Kernel Block and a Multi-Branch Reparameterization Module

Underwater visual detection technology is crucial for marine exploration and monitoring. Given the growing demand for accurate underwater target recognition, this study introduces an innovative architecture, YOLOv8-MU, which significantly enhances the detection accuracy. This model incorporates the large kernel block (LarK block) from UniRepLKNet to optimize the backbone network, achieving a broader receptive field without increasing the model's depth. Additionally, the integration of C2fSTR, which combines the Swin transformer with the C2f module, and the SPPFCSPC_EMA module, which blends Cross-Stage Partial Fast Spatial Pyramid Pooling (SPPFCSPC) with attention mechanisms, notably improves the detection accuracy and robustness for various biological targets. A fusion block from DAMO-YOLO further enhances the multi-scale feature extraction capabilities in the model's neck. Moreover, the adoption of the MPDIoU loss function, designed around the vertex distance, effectively addresses the challenges of localization accuracy and boundary clarity in underwater organism detection. The experimental results on the URPC2019 dataset indicate that YOLOv8-MU achieves an mAP@0.5 of 78.4%, showing an improvement of 4.0% over the original YOLOv8 model. Additionally, on the URPC2020 dataset, it achieves 80.9%, and, on the Aquarium dataset, it reaches 75.5%, surpassing other models, including YOLOv5 and YOLOv8n, thus confirming the wide applicability and generalization capabilities of our proposed improved model architecture. Furthermore, an evaluation on the improved URPC2019 dataset demonstrates leading performance (SOTA), with an mAP@0.5 of 88.1%, further verifying its superiority on this dataset. These results highlight the model's broad applicability and generalization capabilities across various underwater datasets.

Boundary-aware information maximization for self-supervised medical image segmentation

Self-supervised representation learning can boost the performance of a pre-trained network on downstream tasks for which labeled data is limited. A popular method based on this paradigm, known as contrastive learning, works by constructing sets of positive and negative pairs from the data, and then pulling closer the representations of positive pairs while pushing apart those of negative pairs. Although contrastive learning has been shown to improve performance in various classification tasks, its application to image segmentation has been more limited. This stems in part from the difficulty of defining positive and negative pairs for dense feature maps without having access to pixel-wise annotations. In this work, we propose a novel self-supervised pre-training method that overcomes the challenges of contrastive learning in image segmentation. Our method leverages Information Invariant Clustering (IIC) as an unsupervised task to learn a local representation of images in the decoder of a segmentation network, but addresses three important drawbacks of this approach: (i) the difficulty of optimizing the loss based on mutual information maximization; (ii) the lack of clustering consistency for different random transformations of the same image; (iii) the poor correspondence of clusters obtained by IIC with region boundaries in the image. Toward this goal, we first introduce a regularized mutual information maximization objective that encourages the learned clusters to be balanced and consistent across different image transformations. We also propose a boundary-aware loss based on cross-correlation, which helps the learned clusters to be more representative of important regions in the image. Compared to contrastive learning applied in dense features, our method does not require computing positive and negative pairs and also enhances interpretability through the visualization of learned clusters. Comprehensive experiments involving four different medical image segmentation tasks reveal the high effectiveness of our self-supervised representation learning method. Our results show the proposed method to outperform by a large margin several state-of-the-art self-supervised and semi-supervised approaches for segmentation, reaching a performance close to full supervision with only a few labeled examples.

Learnable weight initialization for volumetric medical image segmentation

Hybrid volumetric medical image segmentation models, combining the advantages of local convolution and global attention, have recently received considerable attention. While mainly focusing on architectural modifications, most existing hybrid approaches still use conventional data-independent weight initialization schemes which restrict their performance due to ignoring the inherent volumetric nature of the medical data. To address this issue, we propose a learnable weight initialization approach that utilizes the available medical training data to effectively learn the contextual and structural cues via the proposed self-supervised objectives. Our approach is easy to integrate into any hybrid model and requires no external training data. Experiments on multi-organ and lung cancer segmentation tasks demonstrate the effectiveness of our approach, leading to state-of-the-art segmentation performance. Our proposed data-dependent initialization approach performs favorably as compared to the Swin-UNETR model pretrained using large-scale datasets on multi-organ segmentation task. Our source code and models are available at: https://github.com/ShahinaKK/LWI-VMS.

A lightweight bladder tumor segmentation method based on attention mechanism

In the endoscopic images of bladder, accurate segmentation of different grade bladder tumor from blurred boundary regions and highly variable shapes is of great significance for doctors' diagnosis and patients' later treatment. We propose a nested attentional feature fusion segmentation network (NAFF-Net) based on the encoder-decoder structure formed by the combination of weighted pyramid pooling module (WPPM) and nested attentional feature fusion (NAFF). Among them, WPPM applies the cascade of atrous convolution to enhance the overall perceptual field while introducing adaptive weights to optimize multi-scale feature extraction, NAFF integrates deep semantic information into shallow feature maps, effectively focusing on edge and detail information in bladder tumor images. Additionally, a weighted mixed loss function is constructed to alleviate the impact of imbalance between positive and negative sample distribution on segmentation accuracy. Experiments illustrate the proposed NAFF-Net achieves better segmentation results compared to other mainstream models, with a MIoU of 84.05%, MPrecision of 91.52%, MRecall of 90.81%, and F1-score of 91.16%, and also achieves good results on the public datasets Kvasir-SEG and CVC-ClinicDB. Compared to other models, NAFF-Net has a smaller number of parameters, which is a significant advantage in model deployment.

Robust Ensemble of Two Different Multimodal Approaches to Segment 3D Ischemic Stroke Segmentation Using Brain Tumor Representation Among Multiple Center Datasets

Ischemic stroke segmentation at an acute stage is vital in assessing the severity of patients' impairment and guiding therapeutic decision-making for reperfusion. Although many deep learning studies have shown attractive performance in medical segmentation, it is difficult to use these models trained on public data with private hospitals' datasets. Here, we demonstrate an ensemble model that employs two different multimodal approaches for generalization, a more effective way to perform on external datasets. First, after we jointly train a segmentation model on diffusion-weighted imaging (DWI) and apparent diffusion coefficient (ADC) MR modalities, the model is inferred on the DWI images. Second, a channel-wise segmentation model is trained by concatenating the DWI and ADC images as input, and then is inferred using both MR modalities. Before training with ischemic stroke data, we utilized BraTS 2021, a public brain tumor dataset, for transfer learning. An extensive ablation study evaluates which strategy learns better representations for ischemic stroke segmentation. In our study, nnU-Net well-known for robustness is selected as our baseline model. Our proposed method is evaluated on three different datasets: the Asan Medical Center (AMC) I and II, and the 2022 Ischemic Stroke Lesion Segmentation (ISLES). Our experiments are widely validated over a large, multi-center, and multi-scanner dataset with a huge amount of 846 scans. Not only stroke lesion models can benefit from transfer learning using brain tumor data, but combining the MR modalities using different training schemes also highly improves segmentation performance. The method achieved a top-1 ranking in the ongoing ISLES'22 challenge and performed particularly well on lesion-wise metrics of interest to neuroradiologists, achieving a Dice coefficient of 78.69% and a lesion-wise F1 score of 82.46%. Also, the method was relatively robust on the AMC I (Dice, 60.35%; lesion-wise F1, 68.30%) and II (Dice; 74.12%; lesion-wise F1, 67.53%) datasets in different settings. The high segmentation accuracy of our proposed method could improve radiologists' ability to detect ischemic stroke lesions in MRI images. Our model weights and inference code are available on https://github.com/MDOpx/ISLES22-model-inference .

Diffeomorphic Counterfactuals With Generative Models

Counterfactuals can explain classification decisions of neural networks in a human interpretable way. We propose a simple but effective method to generate such counterfactuals. More specifically, we perform a suitable diffeomorphic coordinate transformation and then perform gradient ascent in these coordinates to find counterfactuals which are classified with great confidence as a specified target class. We propose two methods to leverage generative models to construct such suitable coordinate systems that are either exactly or approximately diffeomorphic. We analyze the generation process theoretically using Riemannian differential geometry and validate the quality of the generated counterfactuals using various qualitative and quantitative measures.

BAL: Balancing Diversity and Novelty for Active Learning

The objective of Active Learning is to strategically label a subset of the dataset to maximize performance within a predetermined labeling budget. In this study, we harness features acquired through self-supervised learning. We introduce a straightforward yet potent metric, Cluster Distance Difference, to identify diverse data. Subsequently, we introduce a novel framework, Balancing Active Learning (BAL), which constructs adaptive sub-pools to balance diverse and uncertain data. Our approach outperforms all established active learning methods on widely recognized benchmarks by 1.20%. Moreover, we assess the efficacy of our proposed framework under extended settings, encompassing both larger and smaller labeling budgets. Experimental results demonstrate that, when labeling 80% of the samples, the performance of the current SOTA method declines by 0.74%, whereas our proposed BAL achieves performance comparable to the full dataset.

Perturbed Progressive Learning for Semisupervised Defect Segmentation

Recently, with the development of intelligent manufacturing, the demand for surface defect inspection has been increasing. Deep learning has achieved promising results in defect inspection. However, due to the rareness of defect data and the difficulties of pixelwise annotation, the existing supervised defect inspection methods are too inferior to be implemented in practice. To solve the problem of defect segmentation with few labeled data, we propose a simple and efficient method for semisupervised defect segmentation (SSDS), named perturbed progressive learning (PPL). On the one hand, PPL decouples the predictions of student and teacher networks as well as alleviates overfitting on noisy pseudo-labels. On the other hand, PPL encourages consistency across various perturbations in a broader stagewise scope, alleviating drift caused by the noisy pseudo-labels. Specifically, PPL contains two training stages. In the first stage, the teacher network gives the unlabeled data with pseudo-labels that are divided into the easy and hard groups. The labeled data and the unlabeled data in the easy group with their perturbation are both used to train for a better-performing student network. In the second stage, the unlabeled data in the hard group are predicted by the obtained student network, so the refined pseudo-labeled data are enlarged. All the pseudo-labeling data and labeled data with their perturbation are used to retrain the student network, progressively improving the defect feature representation. We build a mobile screen defect dataset (MSDD-3) with three classes of defects. PPL is implemented on MSDD-3 as well as other public datasets. Extensive experimental results demonstrate that PPL significantly surpasses the state-of-the-art methods across all evaluation partition protocols.

A fully automatic deep learning-based method for segmenting regions of interest and predicting renal function in pediatric dynamic renal scintigraphy

OBJECTIVE

Accurate delineation of renal regions of interest (ROIs) is critical for the assessment of renal function in pediatric dynamic renal scintigraphy (DRS). The purpose of this study was to develop and evaluate a deep learning (DL) model that can fully automatically delineate renal ROIs and calculate renal function in pediatric 99mTechnetium-ethylenedicysteine (99mTc-EC) DRS.

METHODS

This study retrospectively analyzed 1,283 pediatric DRS data at a single center from January to December 2018. These patients were divided into training set (n = 1027), validation set (n = 128), and testing set (n = 128). A fully automatic segmentation of ROIs (FASR) model was developed and evaluated. The pixel values of the automatically segmented ROIs were calculated to predict renal blood perfusion rate (BPR) and differential renal function (DRF). Precision, recall rate, intersection over union (IOU), and Dice similarity coefficient (DSC) were used to evaluate the performance of FASR model. Intraclass correlation (ICC) and Pearson correlation analysis were used to compare the consistency of automatic and manual method in assessing the renal function parameters in the testing set.

RESULTS

The FASR model achieved a precision of 0.88, recall rate of 0.94, IOU of 0.83, and DSC of 0.91. In the testing set, the r values of BPR and DRF calculated by the two methods were 0.94 (P < 0.01) and 0.97 (P < 0.01), and the ICCs (95% confidence interval CI) were 0.94 (0.90-0.96) and 0.94 (0.91-0.96).

CONCLUSION

We propose a reliable and stable DL model that can fully automatically segment ROIs and accurately predict renal function in pediatric 99mTc-EC DRS.

An Efficient and Rapid Medical Image Segmentation Network

Accurate medical image segmentation is an essential part of the medical image analysis process that provides detailed quantitative metrics. In recent years, extensions of classical networks such as UNet have achieved state-of-the-art performance on medical image segmentation tasks. However, the high model complexity of these networks limits their applicability to devices with constrained computational resources. To alleviate this problem, we propose a shallow hierarchical Transformer for medical image segmentation, called SHFormer. By decreasing the number of transformer blocks utilized, the model complexity of SHFormer can be reduced to an acceptable level. To improve the learned attention while keeping the structure lightweight, we propose a spatial-channel connection module. This module separately learns attention in the spatial and channel dimensions of the feature while interconnecting them to produce more focused attention. To keep the decoder lightweight, the MLP-D module is proposed to progressively fuse multi-scale features in which channels are aligned using Multi-Layer Perceptron (MLP) and spatial information is fused by convolutional blocks. We first validated the performance of SHFormer on the ISIC-2018 dataset. Compared to the latest network, SHFormer exhibits comparable performance with 15 times fewer parameters, 30 times lower computational complexity and 5 times higher inference efficiency. To test the generalizability of SHFormer, we introduced the polyp dataset for additional testing. SHFormer achieves comparable segmentation accuracy to the latest network while having lower computational overhead.

3D Object Detection From Images for Autonomous Driving: A Survey

3D object detection from images, one of the fundamental and challenging problems in autonomous driving, has received increasing attention from both industry and academia in recent years. Benefiting from the rapid development of deep learning technologies, image-based 3D detection has achieved remarkable progress. Particularly, more than 200 works have studied this problem from 2015 to 2021, encompassing a broad spectrum of theories, algorithms, and applications. However, to date no recent survey exists to collect and organize this knowledge. In this paper, we fill this gap in the literature and provide the first comprehensive survey of this novel and continuously growing research field, summarizing the most commonly used pipelines for image-based 3D detection and deeply analyzing each of their components. Additionally, we also propose two new taxonomies to organize the state-of-the-art methods into different categories, with the intent of providing a more systematic review of existing methods and facilitating fair comparisons with future works. In retrospect of what has been achieved so far, we also analyze the current challenges in the field and discuss future directions for image-based 3D detection research.

DeepPyramid+: medical image segmentation using Pyramid View Fusion and Deformable Pyramid Reception

PURPOSE

Semantic segmentation plays a pivotal role in many applications related to medical image and video analysis. However, designing a neural network architecture for medical image and surgical video segmentation is challenging due to the diverse features of relevant classes, including heterogeneity, deformability, transparency, blunt boundaries, and various distortions. We propose a network architecture, DeepPyramid+, which addresses diverse challenges encountered in medical image and surgical video segmentation.

METHODS

The proposed DeepPyramid+ incorporates two major modules, namely "Pyramid View Fusion" (PVF) and "Deformable Pyramid Reception" (DPR), to address the outlined challenges. PVF replicates a deduction process within the neural network, aligning with the human visual system, thereby enhancing the representation of relative information at each pixel position. Complementarily, DPR introduces shape- and scale-adaptive feature extraction techniques using dilated deformable convolutions, enhancing accuracy and robustness in handling heterogeneous classes and deformable shapes.

RESULTS

Extensive experiments conducted on diverse datasets, including endometriosis videos, MRI images, OCT scans, and cataract and laparoscopy videos, demonstrate the effectiveness of DeepPyramid+ in handling various challenges such as shape and scale variation, reflection, and blur degradation. DeepPyramid+ demonstrates significant improvements in segmentation performance, achieving up to a 3.65% increase in Dice coefficient for intra-domain segmentation and up to a 17% increase in Dice coefficient for cross-domain segmentation.

CONCLUSIONS

DeepPyramid+ consistently outperforms state-of-the-art networks across diverse modalities considering different backbone networks, showcasing its versatility. Accordingly, DeepPyramid+ emerges as a robust and effective solution, successfully overcoming the intricate challenges associated with relevant content segmentation in medical images and surgical videos. Its consistent performance and adaptability indicate its potential to enhance precision in computerized medical image and surgical video analysis applications.

Denoised Non-Local Neural Network for Semantic Segmentation

The non-local (NL) network has become a widely used technique for semantic segmentation, which computes an attention map to measure the relationships of each pixel pair. However, most of the current popular NL models tend to ignore the phenomenon that the calculated attention map appears to be very noisy, containing interclass and intraclass inconsistencies, which lowers the accuracy and reliability of the NL methods. In this article, we figuratively denote these inconsistencies as attention noises and explore the solutions to denoise them. Specifically, we inventively propose a denoised NL network, which consists of two primary modules, i.e., the global rectifying (GR) block and the local retention (LR) block, to eliminate the interclass and intraclass noises, respectively. First, GR adopts the class-level predictions to capture a binary map to distinguish whether the selected two pixels belong to the same category. Second, LR captures the ignored local dependencies and further uses them to rectify the unwanted hollows in the attention map. The experimental results on two challenging semantic segmentation datasets demonstrate the superior performance of our model. Without any external training data, our proposed denoised NL can achieve the state-of-the-art performance of 83.5% and 46.69% mean of classwise intersection over union (mIoU) on Cityscapes and ADE20K, respectively.

Deep learning supported echocardiogram analysis: A comprehensive review

An echocardiogram is a sophisticated ultrasound imaging technique employed to diagnose heart conditions. The transthoracic echocardiogram, one of the most prevalent types, is instrumental in evaluating significant cardiac diseases. However, interpreting its results heavily relies on the clinician's expertise. In this context, artificial intelligence has emerged as a vital tool for helping clinicians. This study critically analyzes key state-of-the-art research that uses deep learning techniques to automate transthoracic echocardiogram analysis and support clinical judgments. We have systematically organized and categorized articles that proffer solutions for view classification, enhancement of image quality and dataset, segmentation and identification of cardiac structures, detection of cardiac function abnormalities, and quantification of cardiac functions. We compared the performance of various deep learning approaches within each category, identifying the most promising methods. Additionally, we highlight limitations in current research and explore promising avenues for future exploration. These include addressing generalizability issues, incorporating novel AI approaches, and tackling the analysis of rare cardiac diseases.

Screening and diagnosis of cardiovascular disease using artificial intelligence-enabled cardiac magnetic resonance imaging

Cardiac magnetic resonance imaging (CMR) is the gold standard for cardiac function assessment and plays a crucial role in diagnosing cardiovascular disease (CVD). However, its widespread application has been limited by the heavy resource burden of CMR interpretation. Here, to address this challenge, we developed and validated computerized CMR interpretation for screening and diagnosis of 11 types of CVD in 9,719 patients. We propose a two-stage paradigm consisting of noninvasive cine-based CVD screening followed by cine and late gadolinium enhancement-based diagnosis. The screening and diagnostic models achieved high performance (area under the curve of 0.988 ± 0.3% and 0.991 ± 0.0%, respectively) in both internal and external datasets. Furthermore, the diagnostic model outperformed cardiologists in diagnosing pulmonary arterial hypertension, demonstrating the ability of artificial intelligence-enabled CMR to detect previously unidentified CMR features. This proof-of-concept study holds the potential to substantially advance the efficiency and scalability of CMR interpretation, thereby improving CVD screening and diagnosis.

Multiscale and multiperception feature learning for pancreatic lesion detection based on noncontrast CT

Background. Pancreatic cancer is one of the most malignant tumours, demonstrating a poor prognosis and nearly identically high mortality and morbidity, mainly because of the difficulty of early diagnosis and timely treatment for localized stages.Objective. To develop a noncontrast CT (NCCT)-based pancreatic lesion detection model that could serve as an intelligent tool for diagnosing pancreatic cancer early, overcoming the challenges associated with low contrast intensities and complex anatomical structures present in NCCT images.Approach.We design a multiscale and multiperception (MSMP) feature learning network with ResNet50 coupled with a feature pyramid network as the backbone for strengthening feature expressions. We added multiscale atrous convolutions to expand different receptive fields, contextual attention to perceive contextual information, and channel and spatial attention to focus on important channels and spatial regions, respectively. The MSMP network then acts as a feature extractor for proposing an NCCT-based pancreatic lesion detection model with image patches covering the pancreas as its input; Faster R-CNN is employed as the detection method for accurately detecting pancreatic lesions.Main results. By using the new MSMP network as a feature extractor, our model outperforms the conventional object detection algorithms in terms of the recall (75.40% and 90.95%), precision (40.84% and 68.21%), F1 score (52.98% and 77.96%), F2 score (64.48% and 85.26%) and Ap50 metrics (53.53% and 70.14%) at the image and patient levels, respectively.Significance.The good performance of our new model implies that MSMP can mine NCCT imaging features for detecting pancreatic lesions from complex backgrounds well. The proposed detection model is expected to be further developed as an intelligent method for the early detection of pancreatic cancer.