1
|
Li F, Ma S, Feng Y, Jin C. Research on data consistency detection method based on interactive matching under sampling background. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.109695] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/31/2022]
|
2
|
Song Y, Lu J, Liu A, Lu H, Zhang G. A Segment-Based Drift Adaptation Method for Data Streams. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:4876-4889. [PMID: 33835922 DOI: 10.1109/tnnls.2021.3062062] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
In concept drift adaptation, we aim to design a blind or an informed strategy to update our best predictor for future data at each time point. However, existing informed drift adaptation methods need to wait for an entire batch of data to detect drift and then update the predictor (if drift is detected), which causes adaptation delay. To overcome the adaptation delay, we propose a sequentially updated statistic, called drift-gradient to quantify the increase of distributional discrepancy when every new instance arrives. Based on drift-gradient, a segment-based drift adaptation (SEGA) method is developed to online update our best predictor. Drift-gradient is defined on a segment in the training set. It can precisely quantify the increase of distributional discrepancy between the old segment and the newest segment when only one new instance is available at each time point. A lower value of drift-gradient on the old segment represents that the distribution of the new instance is closer to the distribution of the old segment. Based on the drift-gradient, SEGA retrains our best predictors with the segments that have the minimum drift-gradient when every new instance arrives. SEGA has been validated by extensive experiments on both synthetic and real-world, classification and regression data streams. The experimental results show that SEGA outperforms competitive blind and informed drift adaptation methods.
Collapse
|
3
|
Hoeltgebaum H, Adams N, Fernandes C. Estimation, Forecasting, and Anomaly Detection for Nonstationary Streams Using Adaptive Estimation. IEEE TRANSACTIONS ON CYBERNETICS 2022; 52:7956-7967. [PMID: 33705331 DOI: 10.1109/tcyb.2021.3054161] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Streaming data provides substantial challenges for data analysis. From a computational standpoint, these challenges arise from constraints related to computer memory and processing speed. Statistically, the challenges relate to constructing procedures that can handle the so-called concept drift-the tendency of future data to have different underlying properties to current and historic data. The issue of handling structure, such as trend and periodicity, remains a difficult problem for streaming estimation. We propose the real-time adaptive component (RAC), a penalized-regression modeling framework that satisfies the computational constraints of streaming data, and provides the capability for dealing with concept drift. At the core of the estimation process are techniques from adaptive filtering. The RAC procedure adopts a specified basis to handle local structure, along with a least absolute shrinkage operator-like penalty procedure to handle over fitting. We enhance the RAC estimation procedure with a streaming anomaly detection capability. The experiments with simulated data suggest the procedure can be considered as a competitive tool for a variety of scenarios, and an illustration with real cyber-security data further demonstrates the promise of the method.
Collapse
|
4
|
Wang P, Jin N, Woo WL, Woodward JR, Davies D. Noise tolerant drift detection method for Data Stream Mining. Inf Sci (N Y) 2022. [DOI: 10.1016/j.ins.2022.07.065] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
5
|
Han M, Chen Z, Li M, Wu H, Zhang X. A survey of active and passive concept drift handling methods. Comput Intell 2022. [DOI: 10.1111/coin.12520] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Affiliation(s)
- Meng Han
- School of Computer Science and Engineering North Minzu University Yinchuan China
| | - Zhiqiang Chen
- School of Computer Science and Engineering North Minzu University Yinchuan China
| | - Muhang Li
- School of Computer Science and Engineering North Minzu University Yinchuan China
| | - Hongxin Wu
- School of Computer Science and Engineering North Minzu University Yinchuan China
| | - Xilong Zhang
- School of Computer Science and Engineering North Minzu University Yinchuan China
| |
Collapse
|
6
|
Guo H, Li H, Ren Q, Wang W. Concept drift type identification based on multi-sliding windows. Inf Sci (N Y) 2022. [DOI: 10.1016/j.ins.2021.11.023] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
7
|
Lughofer E, Pratama M. Online sequential ensembling of predictive fuzzy systems. EVOLVING SYSTEMS 2021. [DOI: 10.1007/s12530-021-09398-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Abstract
AbstractEvolving fuzzy systems (EFS) have enjoyed a wide attraction in the community to handle learning from data streams in an incremental, single-pass and transparent manner. The main concentration so far lied in the development of approaches for single EFS models, basically used for prediction purposes. Forgetting mechanisms have been used to increase their flexibility, especially for the purpose to adapt quickly to changing situations such as drifting data distributions. These require forgetting factors steering the degree of timely out-weighing older learned concepts, whose adequate setting in advance or in adaptive fashion is not an easy and not a fully resolved task. In this paper, we propose a new concept of learning fuzzy systems from data streams, which we call online sequential ensembling of fuzzy systems (OS-FS). It is able to model the recent dependencies in streams on a chunk-wise basis: for each new incoming chunk, a new fuzzy model is trained from scratch and added to the ensemble (of fuzzy systems trained before). This induces (i) maximal flexibility in terms of being able to apply variable chunk sizes according to the actual system delay in receiving target values and (ii) fast reaction possibilities in the case of arising drifts. The latter are realized with specific prediction techniques on new data chunks based on the sequential ensemble members trained so far over time. We propose four different prediction variants including various weighting concepts in order to put higher weights on the members with higher inference certainty during the amalgamation of predictions of single members to a final prediction. In this sense, older members, which keep in mind knowledge about past states, may get dynamically reactivated in the case of cyclic drifts, which induce dynamic changes in the process behavior which are re-occurring from time to time later. Furthermore, we integrate a concept for properly resolving possible contradictions among members with similar inference certainties. The reaction onto drifts is thus autonomously handled on demand and on the fly during the prediction stage (and not during model adaptation/evolution stage as conventionally done in single EFS models), which yields enormous flexibility. Finally, in order to cope with large-scale and (theoretically) infinite data streams within a reasonable amount of prediction time, we demonstrate two concepts for pruning past ensemble members, one based on atypical high error trends of single members and one based on the non-diversity of ensemble members. The results based on two data streams showed significantly improved performance compared to single EFS models in terms of a better convergence of the accumulated chunk-wise ahead prediction error trends, especially in the case of regular and cyclic drifts. Moreover, the more advanced prediction schemes could significantly outperform standard averaging over all members’ outputs. Furthermore, resolving contradictory outputs among members helped to improve the performance of the sequential ensemble further. Results on a wider range of data streams from different application scenarios showed (i) improved error trend lines over single EFS models, as well as over related AI methods OS-ELM and MLPs neural networks retrained on data chunks, and (ii) slightly worse trend lines than on-line bagged EFS (as specific EFS ensembles), but with around 100 times faster processing times (achieving low processing times way below requiring milli-seconds for single samples updates).
Collapse
|
8
|
Xuan J, Lu J, Zhang G. Bayesian Nonparametric Unsupervised Concept Drift Detection for Data Stream Mining. ACM T INTEL SYST TEC 2021. [DOI: 10.1145/3420034] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Abstract
Online data stream mining is of great significance in practice because of its ubiquity in many real-world scenarios, especially in the big data era. Traditional data mining algorithms cannot be directly applied to data streams due to (1) the possible change of underlying data distribution over time (i.e.,
concept drift
) and (2) delayed, short, or even no labels for streaming data in practice. A new research area, named
unsupervised concept drift detection
, has emerged to tackle this difficulty mainly based on two-sample hypothesis tests, such as the Kolmogorov–Smirnov test. However, it is surprising that none of the existing methods in this area exploit the Bayesian nonparametric hypothesis test, which has clear interpretability and straightforward prior knowledge encoding ability and no strict or unrealistic requirement of prefixing the form for the underlying data distribution. In this article, we present a Bayesian nonparametric unsupervised concept drift detection method based on the Polya tree hypothesis test. The basic idea is to decompose the underlying data distribution into a multi-resolution representation that transforms the whole distribution hypothesis test into recursive and simple binomial tests. Also, an incremental mechanism is especially designed to improve its efficiency in the stream setting. The method effectively detect drifts, and it also locates where a drift happens and the posteriors of hypotheses. The experiments on synthetic data verify the desired properties of the proposed method, and the experiments on real-world data show the better performance of the method for data stream mining compared with its frequentist counterpart in the literature.
Collapse
Affiliation(s)
- Junyu Xuan
- University of Technology Sydney, Australia
| | - Jie Lu
- University of Technology Sydney, Australia
| | | |
Collapse
|
9
|
|
10
|
Škrjanc I, Iglesias JA, Sanchis A, Leite D, Lughofer E, Gomide F. Evolving fuzzy and neuro-fuzzy approaches in clustering, regression, identification, and classification: A Survey. Inf Sci (N Y) 2019. [DOI: 10.1016/j.ins.2019.03.060] [Citation(s) in RCA: 151] [Impact Index Per Article: 30.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
11
|
Calibration model maintenance in melamine resin production: Integrating drift detection, smart sample selection and model adaptation. Anal Chim Acta 2018; 1013:1-12. [PMID: 29501087 DOI: 10.1016/j.aca.2018.02.003] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2017] [Revised: 01/26/2018] [Accepted: 02/01/2018] [Indexed: 11/23/2022]
Abstract
The physico-chemical properties of Melamine Formaldehyde (MF) based thermosets are largely influenced by the degree of polymerization (DP) in the underlying resin. On-line supervision of the turbidity point by means of vibrational spectroscopy has recently emerged as a promising technique to monitor the DP of MF resins. However, spectroscopic determination of the DP relies on chemometric models, which are usually sensitive to drifts caused by instrumental and/or sample-associated changes occurring over time. In order to detect the time point when drifts start causing prediction bias, we here explore a universal drift detector based on a faded version of the Page-Hinkley (PH) statistic, which we test in three data streams from an industrial MF resin production process. We employ committee disagreement (CD), computed as the variance of model predictions from an ensemble of partial least squares (PLS) models, as a measure for sample-wise prediction uncertainty and use the PH statistic to detect changes in this quantity. We further explore supervised and unsupervised strategies for (semi-)automatic model adaptation upon detection of a drift. For the former, manual reference measurements are requested whenever statistical thresholds on Hotelling's T2 and/or Q-Residuals are violated. Models are subsequently re-calibrated using weighted partial least squares in order to increase the influence of newer samples, which increases the flexibility when adapting to new (drifted) states. Unsupervised model adaptation is carried out exploiting the dual antecedent-consequent structure of a recently developed fuzzy systems variant of PLS termed FLEXFIS-PLS. In particular, antecedent parts are updated while maintaining the internal structure of the local linear predictors (i.e. the consequents). We found improved drift detection capability of the CD compared to Hotelling's T2 and Q-Residuals when used in combination with the proposed PH test. Furthermore, we found that active selection of samples by active learning (AL) used for subsequent model adaptation is advantageous compared to passive (random) selection in case that a drift leads to persistent prediction bias allowing more rapid adaptation at lower reference measurement rates. Fully unsupervised adaptation using FLEXFIS-PLS could improve predictive accuracy significantly for light drifts but was not able to fully compensate for prediction bias in case of significant lack of fit w.r.t. the latent variable space.
Collapse
|
12
|
Learning evolving T–S fuzzy systems with both local and global accuracy – A local online optimization approach. Appl Soft Comput 2018. [DOI: 10.1016/j.asoc.2017.05.046] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
13
|
|
14
|
|
15
|
Lughofer E, Weigl E, Heidl W, Eitzinger C, Radauer T. Recognizing input space and target concept drifts in data streams with scarcely labeled and unlabelled instances. Inf Sci (N Y) 2016. [DOI: 10.1016/j.ins.2016.03.034] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
16
|
de Jesús Rubio J. A method with neural networks for the classification of fruits and vegetables. Soft comput 2016. [DOI: 10.1007/s00500-016-2263-2] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
17
|
Pratama M, Lu J, Lughofer E, Zhang G, Anavatti S. Scaffolding type-2 classifier for incremental learning under concept drifts. Neurocomputing 2016. [DOI: 10.1016/j.neucom.2016.01.049] [Citation(s) in RCA: 28] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
18
|
Hybrid dynamic classifier for drift-like fault diagnosis in a class of hybrid dynamic systems: Application to wind turbine converters. Neurocomputing 2016. [DOI: 10.1016/j.neucom.2015.07.073] [Citation(s) in RCA: 31] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
19
|
|
20
|
Editorial on the special issue “Hybrid and ensemble techniques in soft computing: recent advances and emerging trends”. Soft comput 2015. [DOI: 10.1007/s00500-015-1916-x] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
21
|
Lughofer E, Sayed-Mouchaweh M. Autonomous data stream clustering implementing split-and-merge concepts – Towards a plug-and-play approach. Inf Sci (N Y) 2015. [DOI: 10.1016/j.ins.2015.01.010] [Citation(s) in RCA: 87] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
22
|
Wang L, Wu L, Guan Y, Wang G. Online sensor fault detection based on an improved strong tracking filter. SENSORS 2015; 15:4578-91. [PMID: 25690553 PMCID: PMC4367426 DOI: 10.3390/s150204578] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/02/2014] [Revised: 02/07/2015] [Accepted: 02/09/2015] [Indexed: 11/16/2022]
Abstract
We propose a method for online sensor fault detection that is based on the evolving Strong Tracking Filter (STCKF). The cubature rule is used to estimate states to improve the accuracy of making estimates in a nonlinear case. A residual is the difference in value between an estimated value and the true value. A residual will be regarded as a signal that includes fault information. The threshold is set at a reasonable level, and will be compared with residuals to determine whether or not the sensor is faulty. The proposed method requires only a nominal plant model and uses STCKF to estimate the original state vector. The effectiveness of the algorithm is verified by simulation on a drum-boiler model.
Collapse
Affiliation(s)
- Lijuan Wang
- College of Information Engineering, Capital Normal University, Beijing 100048, China.
- Beijing Engineering Research Center of High Reliable Embedded System, Capital Normal University, Beijing 100048, China.
- Beijing Key Laboratory of Electronic System Reliable Technology, Capital Normal University, Beijing 100048, China.
| | - Lifeng Wu
- College of Information Engineering, Capital Normal University, Beijing 100048, China.
- Beijing Engineering Research Center of High Reliable Embedded System, Capital Normal University, Beijing 100048, China.
- Beijing Key Laboratory of Electronic System Reliable Technology, Capital Normal University, Beijing 100048, China.
| | - Yong Guan
- College of Information Engineering, Capital Normal University, Beijing 100048, China.
- Beijing Engineering Research Center of High Reliable Embedded System, Capital Normal University, Beijing 100048, China.
- Beijing Key Laboratory of Electronic System Reliable Technology, Capital Normal University, Beijing 100048, China.
| | - Guohui Wang
- College of Information Engineering, Capital Normal University, Beijing 100048, China.
- Beijing Engineering Research Center of High Reliable Embedded System, Capital Normal University, Beijing 100048, China.
- Beijing Key Laboratory of Electronic System Reliable Technology, Capital Normal University, Beijing 100048, China.
| |
Collapse
|
23
|
Aloraini A. Penalized ensemble feature selection methods for hidden associations in time series environments case study: equities companies in Saudi Stock Exchange Market. EVOLVING SYSTEMS 2014. [DOI: 10.1007/s12530-014-9124-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|