CASNet: A Cross-Attention Siamese Network for Video Salient Object Detection

PP-NAS: Searching for Plug-and-Play Blocks on Convolutional Neural Networks

Multiscale features are of great importance in modern convolutional neural networks, showing consistent performance gains on numerous vision tasks. Therefore, many plug-and-play blocks are introduced to upgrade existing convolutional neural networks for stronger multiscale representation ability. However, the design of plug-and-play blocks is getting more and more complex, and these manually designed blocks are not optimal. In this work, we propose PP-NAS to develop plug-and-play blocks based on neural architecture search (NAS). Specifically, we design a new search space PPConv and develop a search algorithm consisting of one-level optimization, zero-one loss, and connection existence loss. PP-NAS minimizes the optimization gap between super-net and subarchitectures and can achieve good performance even without retraining. Extensive experiments on image classification, object detection, and semantic segmentation verify the superiority of PP-NAS over state-of-the-art CNNs (e.g., ResNet, ResNeXt, and Res2Net). Our code is available at https://github.com/ainieli/PP-NAS.

Learning Complementary Spatial-Temporal Transformer for Video Salient Object Detection

Besides combining appearance and motion information, another crucial factor for video salient object detection (VSOD) is to mine spatial-temporal (ST) knowledge, including complementary long-short temporal cues and global-local spatial context from neighboring frames. However, the existing methods only explored part of them and ignored their complementarity. In this article, we propose a novel complementary ST transformer (CoSTFormer) for VSOD, which has a short-global branch and a long-local branch to aggregate complementary ST contexts. The former integrates the global context from the neighboring two frames using dense pairwise attention, while the latter is designed to fuse long-term temporal information from more consecutive frames with local attention windows. In this way, we decompose the ST context into a short-global part and a long-local part and leverage the powerful transformer to model the context relationship and learn their complementarity. To solve the contradiction between local window attention and object motion, we propose a novel flow-guided window attention (FGWA) mechanism to align the attention windows with object and camera movements. Furthermore, we deploy CoSTFormer on fused appearance and motion features, thus enabling the effective combination of all three VSOD factors. Besides, we present a pseudo video generation method to synthesize sufficient video clips from static images for training ST saliency models. Extensive experiments have verified the effectiveness of our method and illustrated that we achieve new state-of-the-art results on several benchmark datasets.

Exploring the Temporal Consistency of Arbitrary Style Transfer: A Channelwise Perspective

Arbitrary image stylization by neural networks has become a popular topic, and video stylization is attracting more attention as an extension of image stylization. However, when image stylization methods are applied to videos, unsatisfactory results that suffer from severe flickering effects appear. In this article, we conducted a detailed and comprehensive analysis of the cause of such flickering effects. Systematic comparisons among typical neural style transfer approaches show that the feature migration modules for state-of-the-art (SOTA) learning systems are ill-conditioned and could lead to a channelwise misalignment between the input content representations and the generated frames. Unlike traditional methods that relieve the misalignment via additional optical flow constraints or regularization modules, we focus on keeping the temporal consistency by aligning each output frame with the input frame. To this end, we propose a simple yet efficient multichannel correlation network (MCCNet), to ensure that output frames are directly aligned with inputs in the hidden feature space while maintaining the desired style patterns. An inner channel similarity loss is adopted to eliminate side effects caused by the absence of nonlinear operations such as softmax for strict alignment. Furthermore, to improve the performance of MCCNet under complex light conditions, we introduce an illumination loss during training. Qualitative and quantitative evaluations demonstrate that MCCNet performs well in arbitrary video and image style transfer tasks. Code is available at https://github.com/kongxiuxiu/MCCNetV2.

Artificial intelligence in medical science: a review

Artificial intelligence (AI) is a technique to make intelligent machines, mainly by using smart computer programs. It is based on a statistical analysis of data or machine learning. Using machine learning, software algorithms are designed according to the desired application. These techniques are found to have the potential for advancement in the medical field by generating new and significant perceptions from the data generated using various types of healthcare tests. Artificial intelligence (AI) in medicine is of two types: virtual and physical. The virtual part decides the treatment using electronic health record systems using various sensors whereas the physical part assists robots to perform surgeries, implants, replacement of various organs, elderly care, etc. Using AI, a machine can examine various kinds of health care test reports in one go which could save the time, money, and increase the chances of the patient to be treated without any hassles. At present, artificial intelligence (AI) is used while deciding the treatment, and medications using various tools which could analyze X-rays, CT scans, MRIs, and any other data. During the COVID pandemic, there was a huge/massive demand for AI-supported technologies and many of those were created during that time. This study is focused on various applications of AI in healthcare.

CLRNet: A Cross Locality Relation Network for Crowd Counting in Videos

In this article, we propose a new cross locality relation network (CLRNet) to generate high-quality crowd density maps for crowd counting in videos. Specifically, a cross locality relation module (CLRM) is proposed to enhance feature representations by modeling local dependencies of pixels between adjacent frames with an adapted local self-attention mechanism. First, different from the existing methods which measure similarity between pixels by dot product, a new adaptive cosine similarity is advanced to measure the relationship between two positions. Second, the traditional self-attention modules usually integrate the reconstructed features with the same weights for all the positions. However, crowd movement and background changes in a video sequence are uneven in real-life applications. As a consequence, it is inappropriate to treat all the positions in reconstructed features equally. To address this issue, a scene consistency attention map (SCAM) is developed to make CLRM pay more attention to the positions with strong correlations in adjacent frames. Furthermore, CLRM is incorporated into the network in a coarse-to-fine way to further enhance the representational capability of features. Experimental results demonstrate the effectiveness of our proposed CLRNet in comparison to the state-of-the-art methods on four public video datasets. The codes are available at: https://github.com/Amelie01/CLRNet.

Learning Selective Mutual Attention and Contrast for RGB-D Saliency Detection

How to effectively fuse cross-modal information is a key problem for RGB-D salient object detection. Early fusion and result fusion schemes fuse RGB and depth information at the input and output stages, respectively, and hence incur distribution gaps or information loss. Many models instead employ a feature fusion strategy, but they are limited by their use of low-order point-to-point fusion methods. In this paper, we propose a novel mutual attention model by fusing attention and context from different modalities. We use the non-local attention of one modality to propagate long-range contextual dependencies for the other, thus leveraging complementary attention cues to achieve high-order and trilinear cross-modal interaction. We also propose to induce contrast inference from the mutual attention and obtain a unified model. Considering that low-quality depth data may be detrimental to model performance, we further propose a selective attention to reweight the added depth cues. We embed the proposed modules in a two-stream CNN for RGB-D SOD. Experimental results demonstrate the effectiveness of our proposed model. Moreover, we also construct a new and challenging large-scale RGB-D SOD dataset of high-quality, which can promote both the training and evaluation of deep models.

Optimized Convolutional Neural Network for Road Detection with Structured Contour and Spatial Information for Intelligent Vehicle System

“Road detection is said to be a major research area in remote sensing analysis and it is usually complex due to the data complexities as it gets varied in appearance with minor inter-class and huge intra-class variations that often cause errors and gaps in the extraction of the road”. Moreover, the majority of supervised learning techniques endure from the high price of manual annotation or inadequate training data. Thereby, this paper intends to introduce a new model for road detection. This work exploits a siamesed fully convolutional network (named as “s-FCN-loc”) based on VGG-net architecture that considers semantic contour, RGB channel and location prior for segmenting road regions precisely. As a major contribution, super pixel segmentation was carried out, where the RGB images are given as input to the FCN network and the road regions of images are set as a target. Further, the segmented outputs are fused using AND operation to attain the final segmented output that detects the road regions accurately. To make the detection more accurate, the convolutional layers of FCN are optimally chosen by a new improved model termed as distance oriented sea lion algorithm (DSLnO) model. The presented [Formula: see text] model has achieved a minimal value of negative measures and accuracy is 8.2% higher than traditional methods. Finally, the presented method is evaluated on the KITTI road detection dataset, and achieves a better result. The analysis was done with respect to positive measures and negative measures.

Applying Hybrid Deep Neural Network for the Recognition of Sign Language Words Used by the Deaf COVID-19 Patients

The rapid spread of the novel corona virus disease (COVID-19) has disrupted the traditional clinical services all over the world. Hospitals and healthcare centers have taken extreme care to minimize the risk of exposure to the virus by restricting the visitors and relatives of the patients. The dramatic changes happened in the healthcare norms have made it hard for the deaf patients to communicate and receive appropriate care. This paper reports a work on automatic sign language recognition that can mitigate the communication barrier between the deaf patients and the healthcare workers in India. Since hand gestures are the most expressive components of a sign language vocabulary, a novel dataset of dynamic hand gestures for the Indian sign language (ISL) words commonly used for emergency communication by deaf COVID-19 positive patients is proposed. A hybrid model of deep convolutional long short-term memory network has been utilized for the recognition of the proposed hand gestures and achieved an average accuracy of 83.36%. The model performance has been further validated on an alternative ISL dataset as well as a benchmarking hand gesture dataset and obtained average accuracies of 97 % and 99.34 ± 0.66 % , respectively.

Multi-Sequence MRI Registration of Atherosclerotic Carotid Arteries Based on Cross-Scale Siamese Network

Cardiovascular disease (CVD) is a common disease with high mortality rate, and carotid atherosclerosis (CAS) is one of the leading causes of cardiovascular disease. Multisequence carotid MRI can not only identify carotid atherosclerotic plaque constituents with high sensitivity and specificity, but also obtain different morphological features, which can effectively help doctors improve the accuracy of diagnosis. However, it is difficult to evaluate the accurate evolution of local changes in carotid atherosclerosis in multi-sequence MRI due to the inconsistent parameters of different sequence images and the geometric space mismatch caused by the motion deviation of tissues and organs. To solve these problems, we propose a cross-scale multi-modal image registration method based on the Siamese U-Net. The network uses sub-networks with image inputs of different sizes to extract various features, and a special padding module is designed to make the network available for training on cross-scale features. In addition, to improve the registration performance, a multi-scale loss function under Gaussian smoothing is applied for optimization. For the experiments, we have collected a multi-sequence MRI image dataset from 11 patients with carotid atherosclerosis for a retrospective study. We evaluate our overall architectures by cross-validation on our carotid dataset. The experimental results show that our method can generate precise and reliable results with cross-scale multi-sequence inputs and the registration accuracy can be greatly improved by using the Gaussian smoothing loss function. The DSC of our Siamese structure can reach 84.1% on the carotid data set with cross-size input. With the use of GDSC loss, the average DSC can be improved by 5.23%, while the average distance between fixed landmarks and moving landmarks can be decreased by 6.46%.Our code is made publicly available at: https://github.com/MingHan98/Cross-scale-Siamese-Unet.

TGAN: A simple model update strategy for visual tracking via template-guidance attention network

Visual attention has been widely used in various fields of visual tasks in recent years. Recently, visual trackers based on probabilistic discriminative model prediction (PrDiMP) and Siamese box adaptive network (SiamBAN) have attracted much attention due to their excellent performance and high efficiency. However, the target template of the model in both the PrDiMP and SiamBAN is not updated online, and feature vectors of the template image and the search image are independent of each other in the IoU-Net and Siamese frameworks. In this research, we proposed a template-guidance attention network in both the IoU-Net (denoted as TGAN-I) and Siamese (denoted as TGAN-S) frameworks for visual tracking. TGAN-I and TGAN-S can comprehensively utilize the feature information of the template image and search image, and provide an implicit way to update the template. By utilizing a simple template update strategy, the TGAN-I and TGAN-S trackers can be more robust under certain challenging conditions such as occlusion and deformation. Besides, we introduce a channel and spatial attention module in feature maps of the template image and search image for adaptive feature refinement. Deformable convolutional networks are further used to enhance the model generalization capability in various transformations aspect ratios and scales of tracking targets. To verify the effectiveness of the proposed method, we evaluate the TGAN-I and TGAN-S trackers on six benchmarks and achieve state-of-the-art results. In particular, the TGAN-I method outperforms the strong baseline, PrDiMP, by 0.323 → 0.355 and 0.471 → 0.501 of EAO score on VOT2019 and VOT2016, respectively.