1
|
Yan W, Tan L, Meng-Shan L, Sheng S, Jun W, Fu-an W. SaPt-CNN-LSTM-AR-EA: a hybrid ensemble learning framework for time series-based multivariate DNA sequence prediction. PeerJ 2023; 11:e16192. [PMID: 37810796 PMCID: PMC10559882 DOI: 10.7717/peerj.16192] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 09/06/2023] [Indexed: 10/10/2023] Open
Abstract
Biological sequence data mining is hot spot in bioinformatics. A biological sequence can be regarded as a set of characters. Time series is similar to biological sequences in terms of both representation and mechanism. Therefore, in the article, biological sequences are represented with time series to obtain biological time sequence (BTS). Hybrid ensemble learning framework (SaPt-CNN-LSTM-AR-EA) for BTS is proposed. Single-sequence and multi-sequence models are respectively constructed with self-adaption pre-training one-dimensional convolutional recurrent neural network and autoregressive fractional integrated moving average fused evolutionary algorithm. In DNA sequence experiments with six viruses, SaPt-CNN-LSTM-AR-EA realized the good overall prediction performance and the prediction accuracy and correlation respectively reached 1.7073 and 0.9186. SaPt-CNN-LSTM-AR-EA was compared with other five benchmark models so as to verify its effectiveness and stability. SaPt-CNN-LSTM-AR-EA increased the average accuracy by about 30%. The framework proposed in this article is significant in biology, biomedicine, and computer science, and can be widely applied in sequence splicing, computational biology, bioinformation, and other fields.
Collapse
Affiliation(s)
- Wu Yan
- School of Biotechnology, Jiangsu University of Science & Technology, Zhenjiang, China
- School of Mathematics and Computer Science, Gannan Normal University, Ganzhou, Jiangxi, China
- Sericultural Research Institute, Chinese Academy of Agricultural Sciences, Zhenjiang, Jiangsu, China
| | - Li Tan
- College of Physics and Electronic Information, Gannan Normal University, Ganzhou, China
| | - Li Meng-Shan
- College of Physics and Electronic Information, Gannan Normal University, Ganzhou, China
| | - Sheng Sheng
- School of Biotechnology, Jiangsu University of Science & Technology, Zhenjiang, China
- Sericultural Research Institute, Chinese Academy of Agricultural Sciences, Zhenjiang, Jiangsu, China
| | - Wang Jun
- School of Biotechnology, Jiangsu University of Science & Technology, Zhenjiang, China
- Sericultural Research Institute, Chinese Academy of Agricultural Sciences, Zhenjiang, Jiangsu, China
| | - Wu Fu-an
- School of Biotechnology, Jiangsu University of Science & Technology, Zhenjiang, China
- Sericultural Research Institute, Chinese Academy of Agricultural Sciences, Zhenjiang, Jiangsu, China
| |
Collapse
|
2
|
Shen L, Wei Y, Wang Y. GBT: Two-stage transformer framework for non-stationary time series forecasting. Neural Netw 2023; 165:953-970. [PMID: 37453398 DOI: 10.1016/j.neunet.2023.06.044] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2023] [Revised: 06/09/2023] [Accepted: 06/29/2023] [Indexed: 07/18/2023]
Abstract
This paper shows that time series forecasting Transformer (TSFT) suffers from severe over-fitting problem caused by improper initialization method of unknown decoder inputs, especially when handling non-stationary time series. Based on this observation, we propose GBT, a novel two-stage Transformer framework with Good Beginning. It decouples the prediction process of TSFT into two stages, including Auto-Regression stage and Self-Regression stage to tackle the problem of different statistical properties between input and prediction sequences. Prediction results of Auto-Regression stage serve as a 'Good Beginning', i.e., a better initialization for inputs of Self-Regression stage. We also propose the Error Score Modification module to further enhance the forecasting capability of the Self-Regression stage in GBT. Extensive experiments on seven benchmark datasets demonstrate that GBT outperforms SOTA TSFTs (FEDformer, Pyraformer, ETSformer, etc.) and many other forecasting models (SCINet, N-HiTS, etc.) with only canonical attention and convolution while owning less time and space complexity. It is also general enough to couple with these models to strengthen their forecasting capability. The source code is available at: https://github.com/OrigamiSL/GBT.
Collapse
Affiliation(s)
- Li Shen
- Beihang University, RM.807, 8th Dormitory, Dayuncun Residential Quarter, No. 29, Zhichun Road, Beijing 100191, PR China
| | - Yuning Wei
- Beihang University, RM.807, 8th Dormitory, Dayuncun Residential Quarter, No. 29, Zhichun Road, Beijing 100191, PR China.
| | - Yangzhu Wang
- Beihang University, RM.807, 8th Dormitory, Dayuncun Residential Quarter, No. 29, Zhichun Road, Beijing 100191, PR China
| |
Collapse
|
3
|
Wang X, Liu H, Yang Z, Du J, Dong X. CNformer: a convolutional transformer with decomposition for long-term multivariate time series forecasting. APPL INTELL 2023. [DOI: 10.1007/s10489-023-04496-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/07/2023]
|
4
|
Wang X, Liu H, Du J, Dong X, Yang Z. A long-term multivariate time series forecasting network combining series decomposition and convolutional neural networks. Appl Soft Comput 2023. [DOI: 10.1016/j.asoc.2023.110214] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/29/2023]
|