Artificial intelligence powered advancements in upper extremity joint MRI: A review

Magnetic resonance imaging (MRI) is an indispensable medical imaging examination technique in musculoskeletal medicine. Modern MRI techniques achieve superior high-quality multiplanar imaging of soft tissue and skeletal pathologies without the harmful effects of ionizing radiation. Some current limitations of MRI include long acquisition times, artifacts, and noise. In addition, it is often challenging to distinguish abutting or closely applied soft tissue structures with similar signal characteristics. In the past decade, Artificial Intelligence (AI) has been widely employed in musculoskeletal MRI to help reduce the image acquisition time and improve image quality. Apart from being able to reduce medical costs, AI can assist clinicians in diagnosing diseases more accurately. This will effectively help formulate appropriate treatment plans and ultimately improve patient care. This review article intends to summarize AI's current research and application in musculoskeletal MRI, particularly the advancement of DL in identifying the structure and lesions of upper extremity joints in MRI images.

Deep transfer learning with fuzzy ensemble approach for the early detection of breast cancer

Breast Cancer is a significant global health challenge, particularly affecting women with higher mortality compared with other cancer types. Timely detection of such cancer types is crucial, and recent research, employing deep learning techniques, shows promise in earlier detection. The research focuses on the early detection of such tumors using mammogram images with deep-learning models. The paper utilized four public databases where a similar amount of 986 mammograms each for three classes (normal, benign, malignant) are taken for evaluation. Herein, three deep CNN models such as VGG-11, Inception v3, and ResNet50 are employed as base classifiers. The research adopts an ensemble method where the proposed approach makes use of the modified Gompertz function for building a fuzzy ranking of the base classification models and their decision scores are integrated in an adaptive manner for constructing the final prediction of results. The classification results of the proposed fuzzy ensemble approach outperform transfer learning models and other ensemble approaches such as weighted average and Sugeno integral techniques. The proposed ResNet50 ensemble network using the modified Gompertz function-based fuzzy ranking approach provides a superior classification accuracy of 98.986%.

Application of the convolution neural network in determining the depth of invasion of gastrointestinal cancer: a systematic review and meta-analysis

BACKGROUND

With the development of endoscopic technology, endoscopic submucosal dissection (ESD) has been widely used in the treatment of gastrointestinal tumors. It is necessary to evaluate the depth of tumor invasion before the application of ESD. The convolution neural network (CNN) is a type of artificial intelligence that has the potential to assist in the classification of the depth of invasion in endoscopic images. This meta-analysis aimed to evaluate the performance of CNN in determining the depth of invasion of gastrointestinal tumors.

METHODS

A search on PubMed, Web of Science, and SinoMed was performed to collect the original publications about the use of CNN in determining the depth of invasion of gastrointestinal neoplasms. Pooled sensitivity and specificity were calculated using an exact binominal rendition of the bivariate mixed-effects regression model. I2 was used for the evaluation of heterogeneity.

RESULTS

A total of 17 articles were included; the pooled sensitivity was 84% (95% CI, 0.81-0.88), specificity was 91% (95% CI, 0.85-0.94), and the area under the curve (AUC) was 0.93 (95% CI, 0.90-0.95). The performance of CNN was significantly better than that of endoscopists (AUC: 0.93 vs 0.83, respectively; P = .0005).

CONCLUSION

Our review revealed that CNN is one of the most effective methods of endoscopy to evaluate the depth of invasion of early gastrointestinal tumors, which has the potential to work as a remarkable tool for clinical endoscopists to make decisions on whether the lesion is feasible for endoscopic treatment.

Image detection of aortic dissection complications based on multi-scale feature fusion

Background

Aortic dissection refers to the true and false two-lumen separation of the aortic wall, in which the blood in the aortic lumen enters the aortic mesomembrane from the tear of the aortic intima to separate the mesomembrane and expand along the long axis of the aorta.

Purpose

In view of the problems of individual differences, complex complications and many small targets in clinical aortic dissection detection, this paper proposes a convolution neural network MFF-FPN (Multi-scale Feature Fusion based Feature Pyramid Network) for the detection of aortic dissection complications.

Methods

The proposed model uses Resnet50 as the backbone for feature extraction and builds a pyramid structure to fuse low-level and high-level feature information. We add an attention mechanism to the backbone network, which can establish inter-dependencies between feature graph channels and enhance the representation quality of CNN.

Results

The proposed method has a mean average precision (MAP) of 99.40% in the task of multi object detection for aortic dissection and complications, which is higher than the accuracy of 96.3% on SSD model and 99.05% on YoloV7 model. It greatly improves the accuracy of small target detection such as cysts, making it more suitable for clinical focus detection.

Conclusions

The proposed deep learning model achieves feature reuse and focuses on local important information. By adding only a small number of model parameters, we are able to greatly improve the detection accuracy, which is effective in detecting small target lesions commonly found in clinical settings, and also performs well on other medical and natural datasets.

Peak ground acceleration prediction for on-site earthquake early warning with deep learning

Rapid and accurate prediction of peak ground acceleration (PGA) is an important basis for determining seismic damage through on-site earthquake early warning (EEW). The current on-site EEW uses the feature parameters of the first arrival P-wave to predict PGA, but the selection of these feature parameters is limited by human experience, which limits the accuracy and timeliness of predicting peak ground acceleration (PGA). Therefore, an end-to-end deep learning model is proposed for predicting PGA (DLPGA) based on convolutional neural networks (CNNs). In DLPGA, the vertical initial arrival 3-6 s seismic wave from a single station is used as input, and PGA is used as output. Features are automatically extracted through a multilayer CNN to achieve rapid PGA prediction. The DLPGA is trained, verified, and tested using Japanese seismic records. It is shown that compared to the widely used peak displacement (Pd) method, the correlation coefficient of DLPGA for predicting PGA has increased by 12-23%, the standard deviation of error has decreased by 22-25%, and the error mean has decreased by 6.92-19.66% with the initial 3-6 s seismic waves. In particular, the accuracy of DLPGA for predicting PGA with the initial 3 s seismic wave is better than that of Pd for predicting PGA with the initial 6 s seismic wave. In addition, using the generalization test of Chilean seismic records, it is found that DLPGA has better generalization ability than Pd, and the accuracy of distinguishing ground motion destructiveness is improved by 35-150%. These results confirm that DLPGA has significant accuracy and timeliness advantages over artificially defined feature parameters in predicting PGA, which can greatly improve the effect of on-site EEW in judging the destructiveness of ground motion.

HmsU-Net: A hybrid multi-scale U-net based on a CNN and transformer for medical image segmentation

Accurate medical image segmentation is of great significance for subsequent diagnosis and analysis. The acquisition of multi-scale information plays an important role in segmenting regions of interest of different sizes. With the emergence of Transformers, numerous networks adopted hybrid structures incorporating Transformers and CNNs to learn multi-scale information. However, the majority of research has focused on the design and composition of CNN and Transformer structures, neglecting the inconsistencies in feature learning between Transformer and CNN. This oversight has resulted in the hybrid network's performance not being fully realized. In this work, we proposed a novel hybrid multi-scale segmentation network named HmsU-Net, which effectively fused multi-scale features. Specifically, HmsU-Net employed a parallel design incorporating both CNN and Transformer architectures. To address the inconsistency in feature learning between CNN and Transformer within the same stage, we proposed the multi-scale feature fusion module. For feature fusion across different stages, we introduced the cross-attention module. Comprehensive experiments conducted on various datasets demonstrate that our approach surpasses current state-of-the-art methods.

The montage method improves the classification of suspected acute ischemic stroke using the convolution neural network and brain MRI

This study investigated the usefulness of the montage method that combines four different magnetic resonance images into one images for automatic acute ischemic stroke (AIS) diagnosis with deep learning method. The montage image was consisted from diffusion weighted image (DWI), fluid attenuated inversion recovery (FLAIR), arterial spin labeling (ASL), and apparent diffusion coefficient (ASL). The montage method was compared with pseudo color map (pCM) which was consisted from FLAIR, ASL and ADC. 473 AIS patients were classified into four categories: mechanical thrombectomy, conservative therapy, hemorrhage, and other diseases. The results showed that the montage image significantly outperformed pCM in terms of accuracy (montage image = 0.76 ± 0.01, pCM = 0.54 ± 0.05) and the area under the curve (AUC) (montage image = 0.94 ± 0.01, pCM = 0.76 ± 0.01). This study demonstrates the usefulness of the montage method and its potential for overcoming the limitations of pCM.

Diagnosis of Alzheimer's disease via optimized lightweight convolution-attention and structural MRI

Alzheimer's disease (AD) poses a substantial public health challenge, demanding accurate screening and diagnosis. Identifying AD in its early stages, including mild cognitive impairment (MCI) and healthy control (HC), is crucial given the global aging population. Structural magnetic resonance imaging (sMRI) is essential for understanding the brain's structural changes due to atrophy. While current deep learning networks overlook voxel long-term dependencies, vision transformers (ViT) excel at recognizing such dependencies in images, making them valuable in AD diagnosis. Our proposed method integrates convolution-attention mechanisms in transformer-based classifiers for AD brain datasets, enhancing performance without excessive computing resources. Replacing multi-head attention with lightweight multi-head self-attention (LMHSA), employing inverted residual (IRU) blocks, and introducing local feed-forward networks (LFFN) yields exceptional results. Training on AD datasets with a gradient-centralized optimizer and Adam achieves an impressive accuracy rate of 94.31% for multi-class classification, rising to 95.37% for binary classification (AD vs. HC) and 92.15% for HC vs. MCI. These outcomes surpass existing AD diagnosis approaches, showcasing the model's efficacy. Identifying key brain regions aids future clinical solutions for AD and neurodegenerative diseases. However, this study focused exclusively on the AD Neuroimaging Initiative (ADNI) cohort, emphasizing the need for a more robust, generalizable approach incorporating diverse databases beyond ADNI in future research.

CNN-based noise reduction for multi-channel speech enhancement system with discrete wavelet transform (DWT) preprocessing

Speech enhancement algorithms are applied in multiple levels of enhancement to improve the quality of speech signals under noisy environments known as multi-channel speech enhancement (MCSE) systems. Numerous existing algorithms are used to filter noise in speech enhancement systems, which are typically employed as a pre-processor to reduce noise and improve speech quality. They may, however, be limited in performing well under low signal-to-noise ratio (SNR) situations. The speech devices are exposed to all kinds of environmental noises which may go up to a high-level frequency of noises. The objective of this research is to conduct a noise reduction experiment for a multi-channel speech enhancement (MCSE) system in stationary and non-stationary environmental noisy situations with varying speech signal SNR levels. The experiments examined the performance of the existing and the proposed MCSE systems for environmental noises in filtering low to high SNRs environmental noises (-10 dB to 20 dB). The experiments were conducted using the AURORA and LibriSpeech datasets, which consist of different types of environmental noises. The existing MCSE (BAV-MCSE) makes use of beamforming, adaptive noise reduction and voice activity detection algorithms (BAV) to filter the noises from speech signals. The proposed MCSE (DWT-CNN-MCSE) system was developed based on discrete wavelet transform (DWT) preprocessing and convolution neural network (CNN) for denoising the input noisy speech signals to improve the performance accuracy. The performance of the existing BAV-MCSE and the proposed DWT-CNN-MCSE were measured using spectrogram analysis and word recognition rate (WRR). It was identified that the existing BAV-MCSE reported the highest WRR at 93.77% for a high SNR (at 20 dB) and 5.64% on average for a low SNR (at -10 dB) for different noises. The proposed DWT-CNN-MCSE system has proven to perform well at a low SNR with WRR of 70.55% and the highest improvement (64.91% WRR) at -10 dB SNR.

[Mental fatigue state recognition method based on convolution neural network and long short-term memory]

The pace of modern life is accelerating, the pressure of life is gradually increasing, and the long-term accumulation of mental fatigue poses a threat to health. By analyzing physiological signals and parameters, this paper proposes a method that can identify the state of mental fatigue, which helps to maintain a healthy life. The method proposed in this paper is a new recognition method of psychological fatigue state of electrocardiogram signals based on convolutional neural network and long short-term memory. Firstly, the convolution layer of one-dimensional convolutional neural network model is used to extract local features, the key information is extracted through pooling layer, and some redundant data is removed. Then, the extracted features are used as input to the long short-term memory model to further fuse the ECG features. Finally, by integrating the key information through the full connection layer, the accurate recognition of mental fatigue state is successfully realized. The results show that compared with traditional machine learning algorithms, the proposed method significantly improves the accuracy of mental fatigue recognition to 96.3%, which provides a reliable basis for the early warning and evaluation of mental fatigue.

Deep learning based ultrasonic reconstruction of rough surface morphology

This paper introduces a methodology to recover the morphology of a complex rough surface from ultrasonic pulse echo measurements with an array of equidistant sensors using the one dimensional convolution neural network (1DCNN). The neural network is trained by the datasets simulated from high-fidelity finite element simulations for surfaces with a range of roughness parameters and is tested on both numerical and real experimental data. To assess the performance of our proposed method, the rough surface reconstruction results from the deep learning approach are compared with those obtained from conventional ultrasonic array imaging methods. Unlike array imaging-based methods that require a large number of sensors (e.g., 128, 64 or 32), the deep learning-based method uses pulse echo signals and can achieve accurate results with much fewer sensors. The developed deep learning approach has the potential to enable low-cost, accurate, and real-time reconstruction of complex surface profiles.

Explainable deep-neural-network supported scheme for tuberculosis detection from chest radiographs

Chest radiographs are examined in typical clinical settings by competent physicians for tuberculosis diagnosis. However, this procedure is time consuming and subjective. Due to the growing usage of machine learning techniques in applied sciences, researchers have begun applying comparable concepts to medical diagnostics, such as tuberculosis screening. In the period of extremely deep neural nets which comprised of hundreds of convolution layers for feature extraction, we create a shallow-CNN for screening of TB condition from Chest X-rays so that the model is able to offer appropriate interpretation for right diagnosis. The suggested model consists of four convolution-maxpooling layers with various hyperparameters that were optimized for optimal performance using a Bayesian optimization technique. The model was reported with a peak classification accuracy, F1-score, sensitivity and specificity of 0.95. In addition, the receiver operating characteristic (ROC) curve for the proposed shallow-CNN showed a peak area under the curve value of 0.976. Moreover, we have employed class activation maps (CAM) and Local Interpretable Model-agnostic Explanations (LIME), explainer systems for assessing the transparency and explainability of the model in comparison to a state-of-the-art pre-trained neural net such as the DenseNet.

Attention gate and dilation U-shaped network (GDUNet): an efficient breast ultrasound image segmentation network with multiscale information extraction

Background

In recent years, computer-aided diagnosis (CAD) systems have played an important role in breast cancer screening and diagnosis. The image segmentation task is the key step in a CAD system for the rapid identification of lesions. Therefore, an efficient breast image segmentation network is necessary for improving the diagnostic accuracy in breast cancer screening. However, due to the characteristics of blurred boundaries, low contrast, and speckle noise in breast ultrasound images, breast lesion segmentation is challenging. In addition, many of the proposed breast tumor segmentation networks are too complex to be applied in practice.

Methods

We developed the attention gate and dilation U-shaped network (GDUNet), a lightweight, breast lesion segmentation model. This model improves the inverted bottleneck, integrating it with tokenized multilayer perceptron (MLP) to construct the encoder. Additionally, we introduce the lightweight attention gate (AG) within the skip connection, which effectively filters noise in low-level semantic information across spatial and channel dimensions, thus attenuating irrelevant features. To further improve performance, we innovated the AG dilation (AGDT) block and embedded it between the encoder and decoder in order to capture critical multiscale contextual information.

Results

We conducted experiments on two breast cancer datasets. The experiment's results show that compared to UNet, GDUNet could reduce the number of parameters by 10 times and the computational complexity by 58 times while providing a double of the inference speed. Moreover, the GDUNet achieved a better segmentation performance than did the state-of-the-art medical image segmentation architecture.

Conclusions

Our proposed GDUNet method can achieve advanced segmentation performance on different breast ultrasound image datasets with high efficiency.

Who might encounter hard-braking while speeding? Analysis for regular speeders using low-frequency taxi trajectories on arterial roads and explainable AI

Regular speeders are those who commit speeding recidivism during a period. Among their speeding behaviors, some occurring in specific scenarios may cause more hazards to road users. Therefore, there is a need to evaluate the driving risks if the regular speeders have different speeding propensities. This study considers speeding-related hard-braking events (SHEs) as a safety surrogate measure and recognizes the regular speeders who encounter at least one SHEs during the study period as risky individuals. To identify speeding behaviors and hard-braking events from low-frequency GPS trajectories, we compare the average travel speed between pairwise adjacent GPS points to the posted speed limit and examine the speed curve and the corresponding travel distance between these GPS points, respectively. Thereafter, a logistic model, XGBoost, and three 1D Convolutional Neural Networks (CNNs) including AlexNet CNN, Mini-AlexNet CNN, and Simple CNN are respectively developed to recognize the regular speeders who encountered SHEs based on their speeding propensities. The proposed Mini-AlexNet CNN achieves a global F1-score of 91% and recall of 90% on the testing data, which are superior to other models. Further, the study uses the Shapley Additive exPlanation (SHAP) framework to visually interpret the contribution of speeding propensities on SHE likelihood. It is found that speeding by 50% or greater for no more than 285 m is the most dangerous kind among all the speeding behaviors. Speeding on roads without bicycle lanes or on roads with roadside parking and excessive accesses increases the probability of encountering SHEs. Based on the analyses, we put forward tailored recommendations that aim to restrict hazard-related speeding behaviors rather than speeding behaviors of all kinds.

Improved prostate cancer diagnosis using a modified ResNet50-based deep learning architecture

Prostate cancer, the most common cancer in men, is influenced by age, family history, genetics, and lifestyle factors. Early detection of prostate cancer using screening methods improves outcomes, but the balance between overdiagnosis and early detection remains debated. Using Deep Learning (DL) algorithms for prostate cancer detection offers a promising solution for accurate and efficient diagnosis, particularly in cases where prostate imaging is challenging. In this paper, we propose a Prostate Cancer Detection Model (PCDM) model for the automatic diagnosis of prostate cancer. It proves its clinical applicability to aid in the early detection and management of prostate cancer in real-world healthcare environments. The PCDM model is a modified ResNet50-based architecture that integrates faster R-CNN and dual optimizers to improve the performance of the detection process. The model is trained on a large dataset of annotated medical images, and the experimental results show that the proposed model outperforms both ResNet50 and VGG19 architectures. Specifically, the proposed model achieves high sensitivity, specificity, precision, and accuracy rates of 97.40%, 97.09%, 97.56%, and 95.24%, respectively.

The shape of things to come: Axisymmetric drop shape analysis using deep learning

HYPOTHESIS

In the traditional approach to Axisymmetric Drop Shape Analysis (ADSA), the determination of surface tension or interfacial tension is constrained by computational speed and image quality. By implementing a machine learning-based approach, particularly using a convolutional neural network (CNN), it is posited that analysis of pendant drop images can be both faster and more accurate.

EXPERIMENTS

A CNN model was trained and used to predict the surface tension of drop images. The performance of our CNN model was compared to the traditional ADSA, i.e. direct numerical integration, in terms of precision, computational speed, and robustness in dealing with images of varying quality. Additionally, the ability of the CNN model to predict other drop properties such as Volume and Surface Area was evaluated.

FINDINGS

Our CNN demonstrated a significant enhancement in experimental fit precision, predicting surface tension with an accuracy of (+/-) 1.22×10-1 mN/m and at a speed of 1.50 ms-1, outpacing the traditional method by more than 5×103 times. The model maintained an average surface tension error of 2.42×10-1 mN/m even for experimental images with challenges such as misalignment and poor focus. The CNN model also demonstrated showcased a high degree of accuracy in determining other drop properties.

Semi-supervised medical image segmentation via feature similarity and reliable-region enhancement

Semantic segmentation is a crucial task in the field of computer vision, and medical image segmentation, as its downstream task, has made significant breakthroughs in recent years. However, the issue of requiring a large number of annotations in medical image segmentation has remained a major challenge. Semi-supervised semantic segmentation has provided a powerful approach to address the annotation problem. Nevertheless, existing semi-supervised semantic segmentation methods in medical images have drawbacks, such as insufficient exploitation of unlabeled data information and inefficient utilization of all pseudo-label information. We introduces a novel segmentation model, the Feature Similarity and Reliable-region Enhancement Network (FSRENet), to overcome these limitations. Firstly, this paper proposes a Feature Similarity Module (FSM), which combines the dense feature prediction ability of true labels for unlabeled images with segmentation features as additional constraints, utilizing the similarity relationship between dense features to constrain segmentation features, and thus fully exploiting the dense feature information of unlabeled data. Additionally, the Reliable-region Enhancement Module (REM) designs a high-confidence network structure by fusing two networks that can learn from each other, forming a triple-network structure. The high-confidence network generates reliable pseudo-labels that further constrain the predictions of the two networks, achieving the goal of enhancing the weight of reliable regions, reducing the noise interference of pseudo-labels, and efficiently utilizing all pseudo-label information. Experimental results on the ACDC and LA datasets demonstrate that the FSRENet model proposed in this paper excels in the task of semi-supervised semantic segmentation of medical images and outperforms the majority of existing methods. Our code is available at: https://github.com/gdghds0/FSRENet-master.

Sen-2 LULC: Land use land cover dataset for deep learning approaches

Land Use Land Cover (LULC) classification is pivotal to sustainable environment and natural resource management. It is critical in planning, monitoring, and management programs at various local and national levels. Monitoring changes in LULC patterns over time is crucial for understanding evolving landscapes. Traditionally, LULC classification has been achieved through satellite data by remote sensing, geographic information system (GIS) techniques, machine learning classifiers, and deep learning models. Semantic segmentation, a technique for assigning land cover classes to individual pixels in an image, is commonly employed for LULC mapping. In recent years, the deep learning revolution, particularly Convolutional Neural Networks (CNNs), has reshaped the field of computer vision and LULC classification. Deep architectures have consistently outperformed traditional methods, offering greater accuracy and efficiency. However, the availability of high-quality datasets has been a limiting factor. Bridging the gap between modern computer vision and remote sensing data analysis can revolutionize our understanding of the environment and drive breakthroughs in urban planning and ecosystem change research. The "Sen-2 LULC Dataset" has been created to facilitate this convergence. This dataset comprises of 213,761 pre-processed 10 m resolution images representing seven LULC classes. These classes encompass water bodies, dense forests, sparse forests, barren land, built-up areas, agricultural land, and fallow land. Importantly, each image may contain multiple coexisting land use and land cover classes, mirroring the real-world complexity of landscapes. The dataset is derived from Sentinel-2 satellite imagery sourced from the Copernicus Open Access Hub (https://scihub.copernicus.eu/) platform. It includes spectral bands B4, B3, and B2, corresponding to red, green, and blue (RGB) channels, and offers a spectral resolution of 10 m. The dataset also provides an equal number of mask images. Structured into six folders, the dataset offers training, testing, and validation sets for images and masks. Researchers across various domains can leverage this resource to advance LULC classification in the context of the Indian region. Additionally, it catalyzes fostering collaboration between remote sensing and computer vision communities, enabling novel insights into environmental dynamics and urban planning challenges.

BTC-fCNN: Fast Convolution Neural Network for Multi-class Brain Tumor Classification

Timely prognosis of brain tumors has a crucial role for powerful healthcare of remedy-making plans. Manual classification of the brain tumors in magnetic resonance imaging (MRI) images is a challenging task, which relies on the experienced radiologists to identify and classify the brain tumor. Automated classification of different brain tumors is significant based on designing computer-aided diagnosis (CAD) systems. Existing classification methods suffer from unsatisfactory performance and/or large computational cost/ time. This paper proposed a fast and efficient classification process, called BTC-fCNN, which is a deep learning-based system to distinguish between different views of three brain tumor types, namely meningioma, glioma, and pituitary tumors. The proposed system's model was applied on MRI images from the Figshare dataset. It consists of 13 layers with few trainable parameters involving convolution layer, 1 × 1 convolution layer, average pooling, fully connected layer, and softmax layer. Five iterations including transfer learning and five-fold cross-validation for retraining are considered to increase the proposed model performance. The proposed model achieved 98.63% average accuracy, using five iterations with transfer learning, and 98.86% using retrained five-fold cross-validation (internal transfer learning between the folds). Various evaluation metrics were measured to evaluate the proposed model, such as precision, F-score, recall, specificity and confusion matrix. The proposed BTC-fCNN model outstrips the state-of-the-art and other well-known convolution neural networks (CNN).

CEGAT: A CNN and enhanced-GAT based on key sample selection strategy for hyperspectral image classification

In recent years, the application of convolutional neural networks (CNNs) and graph convolutional networks (GCNs) in hyperspectral image classification (HSIC) has achieved remarkable results. However, the limited label samples are still a major challenge when using CNN and GCN to classify hyperspectral images. In order to alleviate this problem, a double branch fusion network of CNN and enhanced graph attention network (CEGAT) based on key sample selection strategy is proposed. First, a linear discrimination of spectral inter-class slices (LD_SICS) module is designed to eliminate spectral redundancy of HSIs. Then, a spatial spectral correlation attention (SSCA) module is proposed, which can extract and assign attention weight to the spatial and spectral correlation features. On the graph attention (GAT) branch, the HSI is segmented into some super pixels as input to reduce the amount of network parameters. In addition, an enhanced graph attention (EGAT) module is constructed to enhance the relationship between nodes. Finally, a key sample selection (KSS) strategy is proposed to enable the network to achieve better classification performance with few labeled samples. Compared with other state-of-the-art methods, CEGAT has better classification performance under limited label samples.

MLP-based classification of COVID-19 and skin diseases

Recent years have witnessed a growing interest in neural network-based medical image classification methods, which have demonstrated remarkable performance in this field. Typically, convolutional neural network (CNN) architectures have been commonly employed to extract local features. However, the transformer, a newly emerged architecture, has gained popularity due to its ability to explore the relevance of remote elements in an image through a self-attention mechanism. Despite this, it is crucial to establish not only local connectivity but also remote relationships between lesion features and capture the overall image structure to improve image classification accuracy. Therefore, to tackle the aforementioned issues, this paper proposes a network based on multilayer perceptrons (MLPs) that can learn the local features of medical images on the one hand and capture the overall feature information in both spatial and channel dimensions on the other hand, thus utilizing image features effectively. This paper has been extensively validated on COVID19-CT dataset and ISIC 2018 dataset, and the results show that the method in this paper is more competitive and has higher performance in medical image classification compared with existing methods. This shows that the use of MLP to capture image features and establish connections between lesions is expected to provide novel ideas for medical image classification tasks in the future.

Attention-based dual-path feature fusion network for automatic skin lesion segmentation

Automatic segmentation of skin lesions is a critical step in Computer Aided Diagnosis (CAD) of melanoma. However, due to the blurring of the lesion boundary, uneven color distribution, and low image contrast, resulting in poor segmentation result. Aiming at the problem of difficult segmentation of skin lesions, this paper proposes an Attention-based Dual-path Feature Fusion Network (ADFFNet) for automatic skin lesion segmentation. Firstly, in the spatial path, a Boundary Refinement (BR) module is designed for the output of low-level features to filter out irrelevant background information and retain more boundary details of the lesion area. Secondly, in the context path, a Multi-scale Feature Selection (MFS) module is constructed for high-level feature output to capture multi-scale context information and use the attention mechanism to filter out redundant semantic information. Finally, we design a Dual-path Feature Fusion (DFF) module, which uses high-level global attention information to guide the step-by-step fusion of high-level semantic features and low-level detail features, which is beneficial to restore image detail information and further improve the pixel-level segmentation accuracy of skin lesion. In the experiment, the ISIC 2018 and PH2 datasets are employed to evaluate the effectiveness of the proposed method. It achieves a performance of 0.890/ 0.925 and 0.933 /0.954 on the F1-score and SE index, respectively. Comparative analysis with state-of-the-art segmentation methods reveals that the ADFFNet algorithm exhibits superior segmentation performance.

Visual information processing through the interplay between fine and coarse signal pathways

Object recognition is often viewed as a feedforward, bottom-up process in machine learning, but in real neural systems, object recognition is a complicated process which involves the interplay between two signal pathways. One is the parvocellular pathway (P-pathway), which is slow and extracts fine features of objects; the other is the magnocellular pathway (M-pathway), which is fast and extracts coarse features of objects. It has been suggested that the interplay between the two pathways endows the neural system with the capacity of processing visual information rapidly, adaptively, and robustly. However, the underlying computational mechanism remains largely unknown. In this study, we build a two-pathway model to elucidate the computational properties associated with the interactions between two visual pathways. Specifically, we model two visual pathways using two convolution neural networks: one mimics the P-pathway, referred to as FineNet, which is deep, has small-size kernels, and receives detailed visual inputs; the other mimics the M-pathway, referred to as CoarseNet, which is shallow, has large-size kernels, and receives blurred visual inputs. We show that CoarseNet can learn from FineNet through imitation to improve its performance, FineNet can benefit from the feedback of CoarseNet to improve its robustness to noise; and the two pathways interact with each other to achieve rough-to-fine information processing. Using visual backward masking as an example, we further demonstrate that our model can explain visual cognitive behaviors that involve the interplay between two pathways. We hope that this study gives us insight into understanding the interaction principles between two visual pathways.

Unraveling the complexity of Optical Coherence Tomography image segmentation using machine and deep learning techniques: A review

Optical Coherence Tomography (OCT) is an emerging technology that provides three-dimensional images of the microanatomy of biological tissue in-vivo and at micrometer-scale resolution. OCT imaging has been widely used to diagnose and manage various medical diseases, such as macular degeneration, glaucoma, and coronary artery disease. Despite its wide range of applications, the segmentation of OCT images remains difficult due to the complexity of tissue structures and the presence of artifacts. In recent years, different approaches have been used for OCT image segmentation, such as intensity-based, region-based, and deep learning-based methods. This paper reviews the major advances in state-of-the-art OCT image segmentation techniques. It provides an overview of the advantages and limitations of each method and presents the most relevant research works related to OCT image segmentation. It also provides an overview of existing datasets and discusses potential clinical applications. Additionally, this review gives an in-depth analysis of machine learning and deep learning approaches for OCT image segmentation. It outlines challenges and opportunities for further research in this field.

Two-stage contextual transformer-based convolutional neural network for airway extraction from CT images

Accurate airway segmentation from computed tomography (CT) images is critical for planning navigation bronchoscopy and realizing a quantitative assessment of airway-related chronic obstructive pulmonary disease (COPD). Existing methods face difficulty in airway segmentation, particularly for the small branches of the airway. These difficulties arise due to the constraints of limited labeling and failure to meet clinical use requirements in COPD. We propose a two-stage framework with a novel 3D contextual transformer for segmenting the overall airway and small airway branches using CT images. The method consists of two training stages sharing the same modified 3D U-Net network. The novel 3D contextual transformer block is integrated into both the encoder and decoder path of the network to effectively capture contextual and long-range information. In the first training stage, the proposed network segments the overall airway with the overall airway mask. To improve the performance of the segmentation result, we generate the intrapulmonary airway branch label, and train the network to focus on producing small airway branches in the second training stage. Extensive experiments were performed on in-house and multiple public datasets. Quantitative and qualitative analyses demonstrate that our proposed method extracts significantly more branches and longer lengths of the airway tree while accomplishing state-of-the-art airway segmentation performance. The code is available at https://github.com/zhaozsq/airway_segmentation.

AHI estimation of OSAHS patients based on snoring classification and fusion model

Obstructive sleep apnea-hypopnea syndrome (OSAHS) is a chronic and common sleep-breathing disease that could negatively influence lives of patients and cause serious concomitant diseases. Polysomnography(PSG) is the gold standard for diagnosing OSAHS, but it is expensive and requires overnight hospitalization. Snoring is a typical symptom of OSAHS. This study proposes an effective OSAHS screening method based on snoring sound analysis. Snores were labeled as OSAHS related snoring sounds and simple snoring sounds according to real-time PSG records. Three models were used, including acoustic features combined with XGBoost, Mel-spectrum combined with convolution neural network (CNN), and Mel-spectrum combined with residual neural network (ResNet). Further, the three models were fused by soft voting to detect these two types of snoring sounds. The subject's apnea-hypopnea index (AHI) was estimated according to these recognized snoring sounds. The accuracy and recall of the proposed fusion model achieved 83.44% and 85.27% respectively, and the predicted AHI has a Pearson correlation coefficient of 0.913 (R2 = 0.834, p < 0.001) with PSG. The results demonstrate the validity of predicting AHI based on analysis of snoring sound and show great potential for monitoring OSAHS at home.

Lightweight image super-resolution based multi-order gated aggregation network

Recently, Transformer-based models are taken much focus on solving the task of image super-resolution (SR) due to their ability to achieve better performance. However, these models combined huge computational cost during the computing self-attention mechanism. To solve this problem, we proposed a multi-order gated aggregation super-resolution network (MogaSRN) for low-level vision based on the concept of the MogaNet that is developed for high-level vision. The concept of the MogaSRN model is based on spatial multi-order context aggregation and adaptive channel-wise reallocation with the aid of the multi-layer perceptron (MLP). In contrast to the MogaNet model, in which the resolution of each stage decreased by a factor of 2, the resolution of the MogaSRN is stayed fixed during the deep features extraction. Moreover, the structure of the MogaSRN model is built based on balancing the performance and the model complexity. We evaluated our model based on five benchmark datasets concluding that the MogaSRN model can achieve significant improvements compared to the state-of-the-art. Moreover, our model shows the good visual quality and accuracy of the reconstruction. Finally, our model has 3.7 × faster runtime at the scale of × 4 compared to LWSwinIR with better performance.

Convolutional neural network-based recognition method for volleyball movements

With the development of network technology and computer intelligent monitoring technology, a large number of video data came into being. In view of the analysis of specific targets in video, the traditional artificial analysis method can not meet the existing needs. In volleyball teaching in college physical education, because each student has different movements, intelligent processing of video data has become a key issue. Through collecting and sorting out the relevant research results on behavior recognition, it is found that in the research of deep learning, the algorithm structure applied in this paper is representative, and its strong learning ability, especially compared with the traditional algorithm, can more accurately identify human movements, natural language processing and so on, but the research on volleyball action recognition is still less. Therefore, this paper constructs a data set, improves the convolution neural network model, subsequently, new models are constructed through the neural network structure to improve the accuracy of the nonlinear expression and optimize the content of the input data. In order to more accurately analyze the effectiveness of this algorithm, the new data are obtained by grouping the volleyball games in college sports courses. Compared with the original paper, the accuracy of the improved 3D network is improved by 3.3%-88.5%, and the complexity is reduced by 33.6%.

Automated inter-patient arrhythmia classification with dual attention neural network

BACKGROUND AND OBJECTIVES

Arrhythmia classification based on electrocardiograms (ECG) can enhance clinical diagnostic efficiency. However, due to the significant differences in the number of different categories of heartbeats, the performance of classes with fewer samples in arrhythmia classification have not met expectations under the inter-patient paradigm. This paper aims to mitigate the adverse effects of category imbalance and improve arrhythmia classification performance.

METHODS

We constructed a novel dual attention hybrid network (DA-Net) for arrhythmia classification under sample imbalance, based on modified convolutional networks with channel attention (MCC-Net) and sequence-to-sequence network with global attention (Seq2Seq). The refined local features of the input heartbeat are first extracted by MCC-Net and then sent to Seq2Seq for further feature fusion. By applying local and global attention in the feature extraction and fusion parts, respectively, the method fully fuses low-level feature details and high-level context information and enhances the ability to extract discriminative features.

RESULTS

Based on the MIT-BIH arrhythmia database, under the inter-patient paradigm without any data augmentation methods, the proposed method achieved 99.98% accuracy (ACC) for five categories. The various performance indicators are as follows: Class N: sensitivity (SEN) = 99.96%, specificity (SPEC) = 99.93%, positive predictive value (PPV) = 99.99%; Class S: SEN = 99.67%, SPEC = 99.98%, PPV = 99.56%; Class V: SEN = 100%, SPEC = 99.99%, PPV = 99.91%; Class F: SEN = 100%, PPV = 99.98%, SPEC = 97.17%. In further experiments simulating extreme cases, the model still achieved ACC of 99.54% and 98.91% in the three-category and five-category categories when the training sample size was much smaller than the test sample.

CONCLUSIONS

Without any data augmentation methods, the proposed model not only alleviates the negative impact of class imbalance and achieves excellent performance in all categories but also provides a new approach for dealing with class imbalance in arrhythmia classification. Additionally, our method demonstrates potential in conditions with fewer samples.

Prostate lesion segmentation based on a 3D end-to-end convolution neural network with deep multi-scale attention

Prostate cancer is one of the deadest cancers among human beings. To better diagnose the prostate cancer, prostate lesion segmentation becomes a very important work, but its progress is very slow due to the prostate lesions small in size, irregular in shape, and blurred in contour. Therefore, automatic prostate lesion segmentation from mp-MRI is a great significant work and a challenging task. However, the most existing multi-step segmentation methods based on voxel-level classification are time-consuming, may introduce errors in different steps and lead to error accumulation. To decrease the computation time, harness richer 3D spatial features, and fuse the multi-level contextual information of mp-MRI, we present an automatic segmentation method in which all steps are optimized conjointly as one step to form our end-to-end convolutional neural network. The proposed end-to-end network DMSA-V-Net consists of two parts: (1) a 3D V-Net is used as the backbone network, it is the first attempt in employing 3D convolutional neural network for CS prostate lesion segmentation, (2) a deep multi-scale attention mechanism is introduced into the 3D V-Net which can highly focus on the ROI while suppressing the redundant background. As a merit, the attention can adaptively re-align the context information between the feature maps at different scales and the saliency maps in high-levels. We performed experiments based on five cross-fold validation with data including 97 patients. The results show that the Dice and sensitivity are 0.7014 and 0.8652 respectively, which demonstrates that our segmentation approach is more significant and accurate compared to other methods.

Deep learning algorithm for predicting subacromial motion trajectory: Dynamic shoulder ultrasound analysis

Subacromial motion metrics can be extracted from dynamic shoulder ultrasonography, which is useful for identifying abnormal motion patterns in painful shoulders. However, frame-by-frame manual labeling of anatomical landmarks in ultrasound images is time consuming. The present study aims to investigate the feasibility of a deep learning algorithm for extracting subacromial motion metrics from dynamic ultrasonography. Dynamic ultrasound imaging was retrieved by asking 17 participants to perform cyclic shoulder abduction and adduction along the scapular plane, whereby the trajectory of the humeral greater tubercle (in relation to the lateral acromion) was depicted by the deep learning algorithm. Extraction of the subacromial motion metrics was conducted using a convolutional neural network (CNN) or a self-transfer learning-based (STL)-CNN with or without an autoencoder (AE). The mean absolute error (MAE) compared with the manually-labeled data (ground truth) served as the main outcome variable. Using eight-fold cross-validation, the average MAE was proven to be significantly higher in the group using CNN than in those using STL-CNN or STL-CNN+AE for the relative difference between the greater tubercle and lateral acromion on the horizontal axis. The MAE for the localization of the two aforementioned landmarks on the vertical axis also seemed to be enlarged in those using CNN compared with those using STL-CNN. In the testing dataset, the errors in relation to the ground truth for the minimal vertical acromiohumeral distance were 0.081-0.333 cm using CNN, compared with 0.002-0.007 cm using STL-CNN. We successfully demonstrated the feasibility of a deep learning algorithm for automatic detection of the greater tubercle and lateral acromion during dynamic shoulder ultrasonography. Our framework also demonstrated the capability of capturing the minimal vertical acromiohumeral distance, which is the most important indicator of subacromial motion metrics in daily clinical practice.

An automatic improved facial expression recognition for masked faces

Automatic facial expression recognition (AFER), sometimes referred to as emotional recognition, is important for socializing. Automatic methods in the past two years faced challenges due to Covid-19 and the vital wearing of a mask. Machine learning techniques tremendously increase the amount of data processed and achieved good results in such AFER to detect emotions; however, those techniques are not designed for masked faces and thus achieved poor recognition. This paper introduces a hybrid convolutional neural network aided by a local binary pattern to extract features in an accurate way, especially for masked faces. The basic seven emotions classified into anger, happiness, sadness, surprise, contempt, disgust, and fear are to be recognized. The proposed method is applied on two datasets: the first represents CK and CK +, while the second represents M-LFW-FER. Obtained results show that emotion recognition with a face mask achieved an accuracy of 70.76% on three emotions. Results are compared to existing techniques and show significant improvement.

A novel algorithm for small object detection based on YOLOv4

Small object detection is one of the difficulties in the development of computer vision, especially in the case of complex image backgrounds, and the accuracy of small object detection still needs to be improved. In this article, we present a small object detection network based on YOLOv4, which solves some obstacles that hinder the performance of traditional methods in small object detection tasks in complex road environments, such as few effective features, the influence of image noise, and occlusion by large objects, and improves the detection of small objects in complex background situations such as drone aerial survey images. The improved network architecture reduces the computation and GPU memory consumption of the network by including the cross-stage partial network (CSPNet) structure into the spatial pyramid pool (SPP) structure in the YOLOv4 network and convolutional layers after concatenation operation. Secondly, the accuracy of the model on the small object detection task is improved by adding a more suitable small object detection head and removing one used for large object detection. Then, a new branch is added to extract feature information at a shallow location in the backbone part, and the feature information extracted from this branch is fused in the neck part to enrich the small object location information extracted by the model; when fusing feature information from different levels in the backbone, the fusion weight of useful information is increased by adding a weighting mechanism to improve detection performance at each scale. Finally, a coordinated attention (CA) module is embedded at a suitable location in the neck part, which enables the model to focus on spatial location relationships and inter-channel relationships and enhances feature representation capability. The proposed model has been tested to detect 10 different target objects in aerial images from drones and five different road traffic signal signs in images taken from vehicles in a complex road environment. The detection speed of the model meets the criteria of real-time detection, the model has better performance in terms of accuracy compared to the existing state-of-the-art detection models, and the model has only 44M parameters. On the drone aerial photography dataset, the average accuracy of YOLOv4 and YOLOv5L is 42.79% and 42.10%, respectively, while our model achieves an average accuracy (mAP) of 52.76%; on the urban road traffic light dataset, the proposed model achieves an average accuracy of 96.98%, which is also better than YOLOv4 (95.32%), YOLOv5L (94.79%) and other advanced models. The current work provides an efficient method for small object detection in complex road environments, which can be extended to scenarios involving small object detection, such as drone cruising and autonomous driving.

MG-Net: Multi-level global-aware network for thymoma segmentation

BACKGROUND AND OBJECTIVE

Automatic thymoma segmentation in preoperative contrast-enhanced computed tomography (CECT) images makes great sense for diagnosis. Although convolutional neural networks (CNNs) are distinguished in medical image segmentation, they are challenged by thymomas with various shapes, scales and textures, owing to the intrinsic locality of convolution operations. In order to overcome this deficit, we built a deep learning network with enhanced global-awareness for thymoma segmentation.

METHODS

We propose a multi-level global-aware network (MG-Net) for thymoma segmentation, in which the multi-level feature interaction and integration are jointly designed to enhance the global-awareness of CNNs. Particularly, we design the cross-attention block (CAB) to calculate pixel-wise interactions of multi-level features, resulting in the Global Enhanced Convolution Block, which can enable the network to handle various thymomas by strengthening the global-awareness of the encoder. We further devise the Global Spatial Attention Module to integrate coarse- and fine-grain information for enhancing the semantic consistency between the encoder and decoder with CABs. We also develop an Adaptive Attention Fusion Module to adaptively aggregate different semantic-scale features in the decoder to preserve comprehensive details.

RESULTS

The MG-Net has been evaluated against several state-of-the-art models on the self-collected CECT dataset and NIH Pancreas-CT dataset. Results suggest that all designed components are effective, and MG-Net has superior segmentation performance and generalization ability over existing models.

CONCLUSION

Both the qualitative and quantitative experimental results indicate that our MG-Net with global-aware ability can achieve accurate thymoma segmentation and has generalization ability in different tasks. The code is available at: https://github.com/Leejyuan/MGNet.

Compatible-domain Transfer Learning for Breast Cancer Classification with Limited Annotated Data

Microscopic analysis of breast cancer images is the primary task in diagnosing cancer malignancy. Recent attempts to automate this task have employed deep learning models whose success has depended on large volumes of data, while acquiring annotated data in biomedical domains is time-consuming and may not always be feasible. A typical strategy to address this is to apply transfer learning using pre-trained models on a large natural image database (e.g., ImageNet) instead of training a model from scratch. This approach, however, has not been effective in several previous studies due to fundamental differences between natural and medical images. In this study, for the first time we proposed the idea of using a compatible data set of histopathological images to classify breast cancer cytological biopsy specimens. Despite intrinsic differences between histopathological and cytological images, we demonstrate that the features learned by deep networks during the pre-training procedure are compatible with those obtained throughout fine-tuning process. To thoroughly investigate this assertion, we explore three different strategies for training as well as two different approaches for fine-tuning deep learning models. By comparing the obtained results with those of previous state-of-the-art research conducted on the same data set, we demonstrate that the proposed method boasts of improved classification accuracy by 6% to 17% compared to the studies which were based on traditional machine learning techniques, and also enhanced accuracy by roughly 7% compared to those who utilized deep learning methods, eventually achieving 98.73% validation accuracy and 94.55% test accuracy. Exploring different training scenarios also revealed that using a compatible dataset has helped to elevate the classification accuracy by 3.0% compared to the typical approach of using ImageNet. Experimental results show that our approach, despite using a very small number of training images, has achieved performance comparable to that of experienced pathologists and has the potential to be applied in clinical settings.

A novel text sentiment analysis system using improved depthwise separable convolution neural networks

Human behavior is greatly affected by emotions. Human behavior can be predicted by classifying emotions. Therefore, mining people's emotional tendencies from text is of great significance for predicting the behavior of target groups and making decisions. The good use of emotion classification technology can produce huge social and economic benefits. However, due to the rapid development of the Internet, the text information generated on the Internet increases rapidly at an unimaginable speed, which makes the previous method of manually classifying texts one-by-one more and more unable to meet the actual needs. In the subject of sentiment analysis, one of the most pressing problems is how to make better use of computer technology to extract emotional tendencies from text data in a way that is both more efficient and accurate. In the realm of text-based sentiment analysis, the currently available deep learning algorithms have two primary issues to contend with. The first is the high level of complexity involved in training the model, and the second is that the model does not take into account all of the aspects of language and does not make use of word vector information. This research employs an upgraded convolutional neural network (CNN) model as a response to these challenges. The goal of this model is to improve the downsides caused by the problems described above. First, the text separable convolution algorithm is used to perform hierarchical convolution on text features to achieve the refined extraction of word vector information and context information. Doing so avoids semantic confusion and reduces the complexity of convolutional networks. Secondly, the text separable convolution algorithm is applied to text sentiment analysis, and an improved CNN is further proposed. Compared with other models, the proposed model shows better performance in text-based sentiment analysis tasks. This study provides great value for text-based sentiment analysis tasks.

Artificial intelligence in anatomical pathology: building a strong foundation for precision medicine

With the convergence of digital pathology (DP) and artificial intelligence (AI), anatomic pathology practice has been experiencing an exciting paradigm shifting. Pathologists will be provided with an augmented ability to improve diagnostic accuracy, efficiency, and consistency. There will be subvisual morphometric features discovered and multiomics data integrated to provide better prognostic and theragnostic information to guide individual patients' management. The perspective for future precision medicine is promising. However, there are many challenges before AI-assisted DP diagnostic workflows can be successfully implemented. Herein, we briefly review some examples of AI application in anatomic pathology with an emphasis on the subspecialty of gastrointestinal pathology and discuss potential challenges for clinical implementation.

Convolution Neural Network for Breast Cancer Detection and Classification Using Deep Learning

OBJECTIVE

Early detection and precise diagnosis of breast cancer (BC) plays an essential part in enhancing the diagnosis and improving the breast cancer survival rate of patients from 30 to 50%. Through the advances of technology in healthcare, deep learning takes a significant role in handling and inspecting a great number of X-ray, MRI, CTR images.  The aim of this study is to propose a deep learning model (BCCNN) to detect and classify breast cancers into eight classes: benign adenosis (BA), benign fibroadenoma (BF), benign phyllodes tumor (BPT), benign tubular adenoma (BTA), malignant ductal carcinoma (MDC), malignant lobular carcinoma (MLC), malignant mucinous carcinoma (MMC), and malignant papillary carcinoma (MPC).

METHODS

Breast cancer MRI images were classified into BA, BF, BPT, BTA, MDC, MLC, MMC, and MPC using a proposed Deep Learning model with additional 5 fine-tuned Deep learning models consisting of Xception, InceptionV3, VGG16, MobileNet and ResNet50 trained on ImageNet database. The dataset was collected from Kaggle depository for breast cancer detection and classification. That Dataset was boosted using GAN technique. The images in the dataset have 4 magnifications (40X, 100X, 200X, 400X, and Complete Dataset). Thus we evaluated the proposed Deep Learning model and 5 pre-trained models using each dataset individually. That means we carried out a total of 30 experiments. The measurement that was used in the evaluation of all models includes: F1-score, recall, precision, accuracy.

RESULTS

The classification F1-score accuracies of Xception, InceptionV3, ResNet50, VGG16, MobileNet, and Proposed Model (BCCNN) were 97.54%, 95.33%, 98.14%, 97.67%, 93.98%, and 98.28%, respectively.

CONCLUSION

Dataset Boosting, preprocessing and balancing played a good role in enhancing the detection and classification of breast cancer of the proposed model (BCCNN) and the fine-tuned pre-trained models' accuracies greatly. The best accuracies were attained when the 400X magnification of the MRI images due to their high images resolution.

Application of time-frequency domain and deep learning fusion feature in non-invasive diagnosis of congenital heart disease-related pulmonary arterial hypertension

Pulmonary arterial hypertension associated with congenital heart disease (CHD-PAH) is a fatal cardiovascular disease. A novel method for non-invasive initial diagnosis of the CHD-PAH was put forward in this work. First, original heart sounds were segmented into each cardiac cycle by using double-threshold adaptive method. According to clinical auscultation, the pathological information of CHD-PAH is concentrated in S2, so the time-frequency features in both of an entire cardiac cycle and S2 were extracted. Then the time-frequency features combine with the deep learning features to form a feature vector. It is the fusion feature, which will be input into a classifier. Finally, the majority voting algorithm was used to obtain the optimal classification results. A classification accuracy of 88.61% was achieved using this novel method. Three points are essential: •A double-threshold adaptive method is used to segment heart sound into each cardiac cycle.•The time-frequency domain features in both of an entire cardiac cycle and S2 were extracted, which are combined with deep learning features to form the fusion feature.•The XGBoost was used as three-class classifier for the classification of normal, CHD and CHD-PAH. The majority voting algorithm was used to obtain the optimal classification results.

Development of CNN-LSTM combinational architecture for COVID-19 detection

The world has been under extreme pressure due to the spread of the coronavirus. The urgency to eradicate the virus has caused distress amongst civilians and medical agencies to an equal extent. Due to anomalies observed in the results from reverse transcription-polymerase chain reaction (RTPCR) tests, more reliable options like computed tomography (CT) scan-based tests are being researched upon. In this paper, a novel combinational architecture is built upon the principles of Convolution Neural Networks (CNN) and Long Short Term Memory (LSTM) Networks to detect COVID-19 virus. This method uses chest X-ray images as inputs to combinational architecture for the classification of samples. The CNN part of the network will be used to extract features that help in the classification, and the LSTM part will be used for classification based on the extracted features. A total of 8 convolutional layers and 4 pooling layers are used for CNN and 4 LSTM layers of 64 and 128 cells respectively. Instead of the sigmoid function, a rectified linear unit function is used as an activation function. This provides non-linearity to the CNN and better accuracies in comparison. The proposed model employs a padding layer to prevent the loss of information. Accuracy, loss, F1 score, and Matthew's Correlation Coefficient (MCC) are calculated to analyse the effectiveness of the proposed architecture. The proposed model is validated using a relatively larger dataset of 7292 images. The combinational architecture provides a more informative and truthful result in the evaluation of classification as it caters to both the size of positive elements and negative elements in the dataset. The proposed CNN-LSTM model gives an accuracy of 98.91% and an MCC value of 97.84% respectively. The model is also compared with models employing transfer learning methods for similar applications.

Learning effective embedding for automated COVID-19 prediction from chest X-ray images

The pandemic that the SARS-CoV-2 originated in 2019 is continuing to cause serious havoc on the global population's health, economy, and livelihood. A critical way to suppress and restrain this pandemic is the early detection of COVID-19, which will help to control the virus. Chest X-rays are one of the more straightforward ways to detect the COVID-19 virus compared to the standard methods like CT scans and RT-PCR diagnosis, which are very complex, expensive, and take much time. Our research on various papers shows that the currently researchers are actively working for an efficient Deep Learning model to produce an unbiased detection of COVID-19 through chest X-ray images. In this work, we propose a novel convolution neural network model based on supervised classification that simultaneously computes identification and verification loss. We adopt a transfer learning approach using pretrained models trained on imagenet dataset such as Alex Net and VGG16 as back-bone models and use data augmentation techniques to solve class imbalance and boost the classifier's performance. Finally, our proposed classifier architecture model ensures unbiased and high accuracy results, outperforming existing deep learning models for COVID-19 detection from chest X-ray images producing State of the Art performance. It shows strong and robust performance and proves to be easily deployable and scalable, therefore increasing the efficiency of analyzing chest X-ray images with high accuracy in detection of Coronavirus.

An AI-enabled pre-trained model-based Covid detection model using chest X-ray images

The year 2020 and 2021 was the witness of Covid 19 and it was the leading cause of death throughout the world during this time period. It has an impact on a large geographic area, particularly in countries with a large population. Due to the fact that this novel coronavirus has been detected in all countries around the world, the World Health Organization (WHO) has declared Covid-19 to be a pandemic. This novel coronavirus spread quickly from person to person through the saliva droplets and direct or indirect contact with an infected person. The tests carried out to detect the Covid-19 are time-consuming and the primary cause of rapid growth in Covid19 cases. Early detection of Covid patient can play a significant role in controlling the Covid chain by isolation the patient and proper treatment at the right time. Recent research on Covid-19 claim that Chest CT and X-ray images can be used as the preliminary screening for Covid-19 detection. This paper suggested an Artificial Intelligence (AI) based approach for detecting Covid-19 by using X-ray and CT scan images. Due to the availability of the small Covid dataset, we are using a pre-trained model. In this paper, four pre-trained models named VGGNet-19, ResNet50, InceptionResNetV2 and MobileNet are trained to classify the X-ray images into the Covid and Normal classes. A model is tuned in such a way that a smaller percentage of Covid cases will be classified as Normal cases by employing normalization and regularization techniques. The updated binary cross entropy loss (BCEL) function imposes a large penalty for classifying any Covid class to Normal class. The experimental results reveal that the proposed InceptionResNetV2 model outperforms the other pre-trained model with training, validation and test accuracy of 99.2%, 98% and 97% respectively.

Towards precision sleep medicine: Self-attention GAN as an innovative data augmentation technique for developing personalized automatic sleep scoring classification

It is very important to have good quality sleep, which can affect aspects such as memory consolidation, emotional regulation, learning, physical development, and quality of life. Diagnosing human sleep quality and problems quickly and accurately is an important issue for human well-being. Therefore, many automatic sleep scoring methods have been proposed. However, the methods have been developed using sleep data from different individuals or groups. The accuracies of these proposed methods might decrease, due to existing individual differences. In this study, the self-attention generative adversarial network (SAGAN) was applied as an advanced data augmentation technique to propose an improved personalized automatic sleep scoring classification. First, the spectrograms were converted from electroencephalography (EEG). Then, SAGAN was used to generate synthesized spectrograms for each subject. Finally, the real and synthesized spectrograms of each subject were utilized to train a personalized classifier. The averaged accuracy and standard deviation of the proposed method are 95.74% and 3.78%, respectively. Compared to the classifier trained with all subjects' training data, the average accuracy increased by 8.08%. The results proved that the generated spectrograms significantly improved the performance of the personalized automatic sleep scoring classification. The contributions of the proposed method were that made the medical staff and subjects save massive medical resources and time for manual recording and scoring.

LPI-CSFFR: Combining serial fusion with feature reuse for predicting LncRNA-protein interactions

Long non-coding RNAs (LncRNAs) play important roles in a series of life activities, and they function primarily with proteins. The wet experimental-based methods in lncRNA-protein interactions (lncRPIs) study are time-consuming and expensive. In this study, we propose for the first time a novel feature fusion method, the LPI-CSFFR, to train and predict LncRPIs based on a Convolutional Neural Network (CNN) with feature reuse and serial fusion in sequences, secondary structures, and physicochemical properties of proteins and lncRNAs. The experimental results indicate that LPI-CSFFR achieves excellent performance on the datasets RPI1460 and RPI1807 with an accuracy of 83.7 % and 98.1 %, respectively. We further compare LPI-CSFFR with the state-of-the-art existing methods on the same benchmark datasets to evaluate the performance. In addition, to test the generalization performance of the model, we independently test sample pairs of five model organisms, where Mus musculus are the highest prediction accuracy of 99.5 %, and we find multiple hotspot proteins after constructing an interaction network. Finally, we test the predictive power of the LPI-CSFFR for sample pairs with unknown interactions. The results indicate that LPI-CSFFR is promising for predicting potential LncRPIs. The relevant source code and the data used in this study are available at https://github.com/JianjunTan-Beijing/LPI-CSFFR.

Clinical applications of machine learning in the survival prediction and classification of sepsis: coagulation and heparin usage matter

BACKGROUND

Sepsis is a life-threatening syndrome eliciting highly heterogeneous host responses. Current prognostic evaluation methods used in clinical practice are characterized by an inadequate effectiveness in predicting sepsis mortality. Rapid identification of patients with high mortality risk is urgently needed. The phenotyping of patients will assistant invaluably in tailoring treatments.

METHODS

Machine learning and deep learning technology are used to characterize the patients' phenotype and determine the sepsis severity. The database used in this study is MIMIC-III and MIMIC-IV ('Medical information Mart for intensive care') which is a large, public, and freely available database. The K-means clustering is used to classify the sepsis phenotype. Convolutional neural network (CNN) was used to predict the 28-day survival rate based on 35 blood test variables of the sepsis patients, whereas a double coefficient quadratic multivariate fitting function (DCQMFF) is utilized to predict the 28-day survival rate with only 11 features of sepsis patients.

RESULTS

The patients were grouped into four clusters with a clear survival nomogram. The first cluster (C_1) was characterized by low white blood cell count, low neutrophil, and the highest lymphocyte proportion. C_2 obtained the lowest Sequential Organ Failure Assessment (SOFA) score and the highest survival rate. C_3 was characterized by significantly prolonged PTT, high SIC, and a higher proportion of patients using heparin than the patients in other clusters. The early mortality rate of patients in C_3 was high but with a better long-term survival rate than that in C_4. C_4 contained septic coagulation patients with the worst prognosis, characterized by slightly prolonged partial thromboplastin time (PTT), significantly prolonged prothrombin time (PT), and high septic coagulation disease score (SIC). The survival rate prediction accuracy of CNN and DCQMFF models reached 92% and 82%, respectively. The models were tested on an external dataset (MIMIC-IV) and achieved good performance. A DCQMFF-based application platform was established for fast prediction of the 28-day survival rate.

CONCLUSION

CNN and DCQMFF accurately predicted the sepsis patients' survival, while K-means successfully identified the phenotype groups. The distinct phenotypes associated with survival, and significant features correlated with mortality were identified. The findings suggest that sepsis patients with abnormal coagulation had poor outcomes, abnormal coagulation increase mortality during sepsis. The anticoagulation effects of appropriate heparin sodium treatment may improve extensive micro thrombosis-caused organ failure.

Towards more efficient ophthalmic disease classification and lesion location via convolution transformer

OBJECTIVE

A retina optical coherence tomography (OCT) image differs from a traditional image due to its significant speckle noise, irregularity, and inconspicuous features. A conventional deep learning architecture cannot effectively improve the classification accuracy, sensitivity, and specificity of OCT images, and noisy images are not conducive to further diagnosis.  This paper proposes a novel lesion-localization convolution transformer (LLCT) method, which combines both convolution and self-attention to classify ophthalmic diseases more accurately and localize the lesions in retina OCT images.

METHODS

A novel architecture design is accomplished through applying customized feature maps generated by convolutional neutral network (CNN) as the input sequence of self-attention network. This design takes advantages of CNN's extracting image features and transformer's consideration of global context and dynamic attention. Part of the model is backward propagated to calculate the gradient as a weight parameter, which is multiplied and summed with the global features generated by the forward propagation process to locate the lesion.

RESULTS

Extensive experiments show that our proposed design achieves improvement of about 7.6% in overall accuracy, 10.9% in overall sensitivity, and 9.2% in overall specificity compared with previous methods. And the lesions can be localized without the labeling data of lesion location in OCT images.

CONCLUSION

The results prove that our method significantly improves the performance and reduces the computation complexity in artificial intelligence assisted analysis of ophthalmic disease through OCT images.

SIGNIFICANCE

Our method has a significance boost in ophthalmic disease classification and location via convolution transformer. This is applicable to assist ophthalmologists greatly.¹.

CNNLSTMac4CPred: A Hybrid Model for N4-Acetylcytidine Prediction

N4-Acetylcytidine (ac4C) is a highly conserved post-transcriptional and an extensively existing RNA modification, playing versatile roles in the cellular processes. Due to the limitation of techniques and knowledge, large-scale identification of ac4C is still a challenging task. RNA sequences are like sentences containing semantics in the natural language. Inspired by the semantics of language, we proposed a hybrid model for ac4C prediction. The model used long short-term memory and convolution neural network to extract the semantic features hidden in the sequences. The semantic and the two traditional features (k-nucleotide frequencies and pseudo tri-tuple nucleotide composition) were combined to represent ac4C or non-ac4C sequences. The eXtreme Gradient Boosting was used as the learning algorithm. Five-fold cross-validation over the training set consisting of 1160 ac4C and 10,855 non-ac4C sequences obtained the area under the receiver operating characteristic curve (AUROC) of 0.9004, and the independent test over 469 ac4C and 4343 non-ac4C sequences reached an AUROC of 0.8825. The model obtained a sensitivity of 0.6474 in the five-fold cross-validation and 0.6290 in the independent test, outperforming two state-of-the-art methods. The performance of semantic features alone was better than those of k-nucleotide frequencies and pseudo tri-tuple nucleotide composition, implying that ac4C sequences are of semantics. The proposed hybrid model was implemented into a user-friendly web-server which is freely available to scientific communities: http://47.113.117.61/ac4c/ . The presented model and tool are beneficial to identify ac4C on large scale.

Chest Radiographs Using a Context-Fusion Convolution Neural Network (CNN): Can It Distinguish the Etiology of Community-Acquired Pneumonia (CAP) in Children?

Clinical symptoms and inflammatory markers cannot reliably distinguish the etiology of CAP, and chest radiographs have abundant information related with CAP. Hence, we developed a context-fusion convolution neural network (CNN) to explore the application of chest radiographs to distinguish the etiology of CAP in children. This retrospective study included 1769 cases of pediatric pneumonia (viral pneumonia, n = 487; bacterial pneumonia, n = 496; and mycoplasma pneumonia, n = 786). The chest radiographs of the first examination, C-reactive protein (CRP), and white blood cell (WBC) were collected for analysis. All patients were stochastically divided into training, validation, and test cohorts in a 7:1:2 ratio. Automatic lung segmentation and hand-crafted pneumonia lesion segmentation were performed, from which three image-based models including a full-lung model, a local-lesion model, and a context-fusion model were built; two clinical characteristics were used to build a clinical model, while a logistic regression model combined the best CNN model and two clinical characteristics. Our experiments showed that the context-fusion model which integrated the features of the full-lung and local-lesion had better performance than the full-lung model and local-lesion model. The context-fusion model had area under curves of 0.86, 0.88, and 0.93 in identifying viral, bacterial, and mycoplasma pneumonia on the test cohort respectively. The addition of clinical characteristics to the context-fusion model obtained slight improvement. Mycoplasma pneumonia was more easily identified compared with the other two types. Using chest radiographs, we developed a context-fusion CNN model with good performance for noninvasively diagnosing the etiology of community-acquired pneumonia in children, which would help improve early diagnosis and treatment.

iProm-Zea: A two-layer model to identify plant promoters and their types using convolutional neural network

A promoter is a short DNA sequence near the start codon, responsible for initiating the transcription of a specific gene in the genome. The accurate recognition of promoters is important for achieving a better understanding of transcriptional regulation. Because of their importance in the process of biological transcriptional regulation, there is an urgent need to develop in silico tools to identify promoters and their types in a timely and accurate manner. A number of prediction methods have been developed in this regard; however, almost all of them are merely used for identifying promoters and their strength or sigma types. The TATA box region in TATA promoter influences the post-transcriptional processes; therefore, in the current study, we developed a two-layer predictor called "iProm-Zea" using the convolutional neural network (CNN) for identify TATA and TATA less promoters. The first layer can be used to identify a given DNA sequence as a promoter or non-promoter. The second layer can be used to identify whether the recognized promoter is the TATA promoter. To find an optimal feature encoding scheme and model, we employed four feature encoding schemes on different machine learning and CNN algorithms, and based on the evaluation results, we selected a one-hot encoding scheme and a CNN model for iProm-Zea. The 5-fold cross validation testing results demonstrated that the constructed predictor showed great potential for identifying promoters and classifying them as TATA and TATA less promoters. Furthermore, we performed cross-species analysis of iProm-Zea to evaluate its performance in other species. Moreover, to make it easier for other experimental scientists to obtain the results they need, we established a freely accessible and user-friendly web server at http://nsclbio.jbnu.ac.kr/tools/iProm-Zea/.

Deep6mAPred: A CNN and Bi-LSTM-based deep learning method for predicting DNA N6-methyladenosine sites across plant species

DNA N6-methyladenine (6mA) is a key DNA modification, which plays versatile roles in the cellular processes, including regulation of gene expression, DNA repair, and DNA replication. DNA 6mA is closely associated with many diseases in the mammals and with growth as well as development of plants. Precisely detecting DNA 6mA sites is of great importance to exploration of 6mA functions. Although many computational methods have been presented for DNA 6mA prediction, there is still a wide gap in the practical application. We presented a convolution neural network (CNN) and bi-directional long-short term memory (Bi-LSTM)-based deep learning method (Deep6mAPred) for predicting DNA 6mA sites across plant species. The Deep6mAPred stacked the CNNs and the Bi-LSTMs in a paralleling manner instead of a series-connection manner. The Deep6mAPred also employed the attention mechanism for improving the representations of sequences. The Deep6mAPred reached an accuracy of 0.9556 over the independent rice dataset, far outperforming the state-of-the-art methods. The tests across plant species showed that the Deep6mAPred is of a remarkable advantage over the state of the art methods. We developed a user-friendly web application for DNA 6mA prediction, which is freely available at http://106.13.196.152:7001/ for all the scientific researchers. The Deep6mAPred would enrich tools to predict DNA 6mA sites and speed up the exploration of DNA modification.