1
|
Wang Z, Gu J, Zhou W, He Q, Zhao T, Guo J, Lu L, He T, Bu J. Neural Memory State Space Models for Medical Image Segmentation. Int J Neural Syst 2024:2450068. [PMID: 39343431 DOI: 10.1142/s0129065724500680] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/01/2024]
Abstract
With the rapid advancement of deep learning, computer-aided diagnosis and treatment have become crucial in medicine. UNet is a widely used architecture for medical image segmentation, and various methods for improving UNet have been extensively explored. One popular approach is incorporating transformers, though their quadratic computational complexity poses challenges. Recently, State-Space Models (SSMs), exemplified by Mamba, have gained significant attention as a promising alternative due to their linear computational complexity. Another approach, neural memory Ordinary Differential Equations (nmODEs), exhibits similar principles and achieves good results. In this paper, we explore the respective strengths and weaknesses of nmODEs and SSMs and propose a novel architecture, the nmSSM decoder, which combines the advantages of both approaches. This architecture possesses powerful nonlinear representation capabilities while retaining the ability to preserve input and process global information. We construct nmSSM-UNet using the nmSSM decoder and conduct comprehensive experiments on the PH2, ISIC2018, and BU-COCO datasets to validate its effectiveness in medical image segmentation. The results demonstrate the promising application value of nmSSM-UNet. Additionally, we conducted ablation experiments to verify the effectiveness of our proposed improvements on SSMs and nmODEs.
Collapse
Affiliation(s)
- Zhihua Wang
- College of Computer Science, Zhejiang University, Hangzhou, P. R. China
- Zhejiang Provincial Key Laboratory of Service Robot, Hangzhou, Zhejiang Province, P. R. China
| | - Jingjun Gu
- College of Computer Science, Zhejiang University, Hangzhou, P. R. China
- Zhejiang Provincial Key Laboratory of Service Robot, Hangzhou, Zhejiang Province, P. R. China
| | - Wang Zhou
- Department of Ultrasound, The First Affiliated Hospital of Anhui Medical University, Hefei, P. R. China
| | - Quansong He
- College of Computer Science, Sichuan University, Chengdu, P. R. China
| | - Tianli Zhao
- Department of Cardiovascular Surgery, The Second Xiangya Hospital, Central South University, Changsha, P. R. China
| | - Jialong Guo
- College of Computer Science, Zhejiang University, Hangzhou, P. R. China
- Zhejiang Provincial Key Laboratory of Service Robot, Hangzhou, Zhejiang Province, P. R. China
| | - Li Lu
- Department of Ophthalmology, Eye Center, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, P. R. China
| | - Tao He
- College of Computer Science, Sichuan University, Chengdu, P. R. China
| | - Jiajun Bu
- College of Computer Science, Zhejiang University, Hangzhou, P. R. China
- Zhejiang Provincial Key Laboratory of Service Robot, Hangzhou, Zhejiang Province, P. R. China
| |
Collapse
|
2
|
Xu X, Luo H, Yi Z, Zhang H. A Forward Learning Algorithm for Neural Memory Ordinary Differential Equations. Int J Neural Syst 2024; 34:2450048. [PMID: 38909317 DOI: 10.1142/s0129065724500485] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/24/2024]
Abstract
The deep neural network, based on the backpropagation learning algorithm, has achieved tremendous success. However, the backpropagation algorithm is consistently considered biologically implausible. Many efforts have recently been made to address these biological implausibility issues, nevertheless, these methods are tailored to discrete neural network structures. Continuous neural networks are crucial for investigating novel neural network models with more biologically dynamic characteristics and for interpretability of large language models. The neural memory ordinary differential equation (nmODE) is a recently proposed continuous neural network model that exhibits several intriguing properties. In this study, we present a forward-learning algorithm, called nmForwardLA, for nmODE. This algorithm boasts lower computational dimensions and greater efficiency. Compared with the other learning algorithms, experimental results on MNIST, CIFAR10, and CIFAR100 demonstrate its potency.
Collapse
Affiliation(s)
- Xiuyuan Xu
- Department of Computer Science, Sichuan University, No. 24 South Section 1, Yihuan Road, Chengdu 610065 Sichuan, P. R. China
| | - Haiying Luo
- Department of Computer Science, Sichuan University, No. 24 South Section 1, Yihuan Road, Chengdu 610065 Sichuan, P. R. China
| | - Zhang Yi
- Department of Computer Science, Sichuan University, No. 24 South Section 1, Yihuan Road, Chengdu 610065 Sichuan, P. R. China
| | - Haixian Zhang
- Department of Computer Science, Sichuan University, No. 24 South Section 1, Yihuan Road, Chengdu 610065 Sichuan, P. R. China
| |
Collapse
|
3
|
Zheng X, Yang Y, Li D, Deng Y, Xie Y, Yi Z, Ma L, Xu L. Precise Localization for Anatomo-Physiological Hallmarks of the Cervical Spine by Using Neural Memory Ordinary Differential Equation. Int J Neural Syst 2024:2450056. [PMID: 39049777 DOI: 10.1142/s0129065724500564] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/27/2024]
Abstract
In the evaluation of cervical spine disorders, precise positioning of anatomo-physiological hallmarks is fundamental for calculating diverse measurement metrics. Despite the fact that deep learning has achieved impressive results in the field of keypoint localization, there are still many limitations when facing medical image. First, these methods often encounter limitations when faced with the inherent variability in cervical spine datasets, arising from imaging factors. Second, predicting keypoints for only 4% of the entire X-ray image surface area poses a significant challenge. To tackle these issues, we propose a deep neural network architecture, NF-DEKR, specifically tailored for predicting keypoints in cervical spine physiological anatomy. Leveraging neural memory ordinary differential equation with its distinctive memory learning separation and convergence to a singular global attractor characteristic, our design effectively mitigates inherent data variability. Simultaneously, we introduce a Multi-Resolution Focus module to preprocess feature maps before entering the disentangled regression branch and the heatmap branch. Employing a differentiated strategy for feature maps of varying scales, this approach yields more accurate predictions of densely localized keypoints. We construct a medical dataset, SCUSpineXray, comprising X-ray images annotated by orthopedic specialists and conduct similar experiments on the publicly available UWSpineCT dataset. Experimental results demonstrate that compared to the baseline DEKR network, our proposed method enhances average precision by 2% to 3%, accompanied by a marginal increase in model parameters and the floating-point operations (FLOPs). The code (https://github.com/Zhxyi/NF-DEKR) is available.
Collapse
Affiliation(s)
- Xi Zheng
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, No. 24 South Section 1, Yihuan Road, Chengdu 610065, P. R. China
| | - Yi Yang
- Department of Orthopedics, Orthopedic Research Institute, West China Hospital, Sichuan University, No. 37 Guo Xue Road, Chengdu 610041, P. R. China
| | - Dehan Li
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, No. 24 South Section 1, Yihuan Road, Chengdu 610065, P. R. China
| | - Yi Deng
- Department of Orthopedics, Orthopedic Research Institute, West China Hospital, Sichuan University, No. 37 Guo Xue Road, Chengdu 610041, P. R. China
| | - Yuexiong Xie
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, No. 24 South Section 1, Yihuan Road, Chengdu 610065, P. R. China
| | - Zhang Yi
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, No. 24 South Section 1, Yihuan Road, Chengdu 610065, P. R. China
| | - Litai Ma
- Department of Orthopedics, Orthopedic Research Institute, West China Hospital, Sichuan University, No. 37 Guo Xue Road, Chengdu 610041, P. R. China
| | - Lei Xu
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, No. 24 South Section 1, Yihuan Road, Chengdu 610065, P. R. China
| |
Collapse
|
4
|
Niu H, Yi Z, He T. A Bidirectional Feedforward Neural Network Architecture Using the Discretized Neural Memory Ordinary Differential Equation. Int J Neural Syst 2024; 34:2450015. [PMID: 38318709 DOI: 10.1142/s0129065724500151] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2024]
Abstract
Deep Feedforward Neural Networks (FNNs) with skip connections have revolutionized various image recognition tasks. In this paper, we propose a novel architecture called bidirectional FNN (BiFNN), which utilizes skip connections to aggregate features between its forward and backward paths. The BiFNN accepts any FNN as a plugin that can incorporate any general FNN model into its forward path, introducing only a few additional parameters in the cross-path connections. The backward path is implemented as a nonparameter layer, utilizing a discretized form of the neural memory Ordinary Differential Equation (nmODE), which is named [Formula: see text]-net. We provide a proof of convergence for the [Formula: see text]-net and evaluate its initial value problem. Our proposed architecture is evaluated on diverse image recognition datasets, including Fashion-MNIST, SVHN, CIFAR-10, CIFAR-100, and Tiny-ImageNet. The results demonstrate that BiFNNs offer significant improvements compared to embedded models such as ConvMixer, ResNet, ResNeXt, and Vision Transformer. Furthermore, BiFNNs can be fine-tuned to achieve comparable performance with embedded models on Tiny-ImageNet and ImageNet-1K datasets by loading the same pretrained parameters.
Collapse
Affiliation(s)
- Hao Niu
- College of Computer Science, Sichuan University, Chengdu 610065, P. R. China
| | - Zhang Yi
- College of Computer Science, Sichuan University, Chengdu 610065, P. R. China
| | - Tao He
- College of Computer Science, Sichuan University, Chengdu 610065, P. R. China
| |
Collapse
|