1
|
Wang G, Fan F, Shi S, An S, Cao X, Ge W, Yu F, Wang Q, Han X, Tan S, Tan Y, Wang Z. Multi modality fusion transformer with spatio-temporal feature aggregation module for psychiatric disorder diagnosis. Comput Med Imaging Graph 2024; 114:102368. [PMID: 38518412 DOI: 10.1016/j.compmedimag.2024.102368] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Revised: 03/02/2024] [Accepted: 03/13/2024] [Indexed: 03/24/2024]
Abstract
Bipolar disorder (BD) is characterized by recurrent episodes of depression and mild mania. In this paper, to address the common issue of insufficient accuracy in existing methods and meet the requirements of clinical diagnosis, we propose a framework called Spatio-temporal Feature Fusion Transformer (STF2Former). It improves on our previous work - MFFormer by introducing a Spatio-temporal Feature Aggregation Module (STFAM) to learn the temporal and spatial features of rs-fMRI data. It promotes intra-modality attention and information fusion across different modalities. Specifically, this method decouples the temporal and spatial dimensions and designs two feature extraction modules for extracting temporal and spatial information separately. Extensive experiments demonstrate the effectiveness of our proposed STFAM in extracting features from rs-fMRI, and prove that our STF2Former can significantly outperform MFFormer and achieve much better results among other state-of-the-art methods.
Collapse
Affiliation(s)
- Guoxin Wang
- College of Biomedical Engineering & Instrument Science, Zhejiang University, Hangzhou 310027, China
| | - Fengmei Fan
- Beijing Huilongguan Hospital, Peking University Huilongguan Clinical Medical School, Beijing 100096, China
| | - Sheng Shi
- College of Sciences, Northeastern University, Shenyang 110819, China
| | - Shan An
- JD Health International Inc., Beijing 100176, China
| | - Xuyang Cao
- JD Health International Inc., Beijing 100176, China
| | - Wenshu Ge
- JD Health International Inc., Beijing 100176, China
| | - Feng Yu
- College of Biomedical Engineering & Instrument Science, Zhejiang University, Hangzhou 310027, China.
| | - Qi Wang
- College of Sciences, Northeastern University, Shenyang 110819, China.
| | - Xiaole Han
- Beijing Huilongguan Hospital, Peking University Huilongguan Clinical Medical School, Beijing 100096, China
| | - Shuping Tan
- Beijing Huilongguan Hospital, Peking University Huilongguan Clinical Medical School, Beijing 100096, China
| | - Yunlong Tan
- Beijing Huilongguan Hospital, Peking University Huilongguan Clinical Medical School, Beijing 100096, China
| | - Zhiren Wang
- Beijing Huilongguan Hospital, Peking University Huilongguan Clinical Medical School, Beijing 100096, China.
| |
Collapse
|