A systematic literature review of methods and datasets for anomaly-based network intrusion detection

Deep Complex Gated Recurrent Networks-Based IoT Network Intrusion Detection Systems

The explosive growth of the Internet of Things (IoT) has highlighted the urgent need for strong network security measures. The distinctive difficulties presented by Internet of Things (IoT) environments, such as the wide variety of devices, the intricacy of network traffic, and the requirement for real-time detection capabilities, are difficult for conventional intrusion detection systems (IDS) to adjust to. To address these issues, we propose DCGR_IoT, an innovative intrusion detection system (IDS) based on deep neural learning that is intended to protect bidirectional communication networks in the IoT environment. DCGR_IoT employs advanced techniques to enhance anomaly detection capabilities. Convolutional neural networks (CNN) are used for spatial feature extraction and superfluous data are filtered to improve computing efficiency. Furthermore, complex gated recurrent networks (CGRNs) are used for the temporal feature extraction module, which is utilized by DCGR_IoT. Furthermore, DCGR_IoT harnesses complex gated recurrent networks (CGRNs) to construct multidimensional feature subsets, enabling a more detailed spatial representation of network traffic and facilitating the extraction of critical features that are essential for intrusion detection. The effectiveness of the DCGR_IoT was proven through extensive evaluations of the UNSW-NB15, KDDCup99, and IoT-23 datasets, which resulted in a high detection accuracy of 99.2%. These results demonstrate the DCG potential of DCGR-IoT as an effective solution for defending IoT networks against sophisticated cyber-attacks.

End-to-End Network Intrusion Detection Based on Contrastive Learning

The network intrusion detection system (NIDS) plays a crucial role as a security measure in addressing the increasing number of network threats. The majority of current research relies on feature-ready datasets that heavily depend on feature engineering. Conversely, the increasing complexity of network traffic and the ongoing evolution of attack techniques lead to a diminishing distinction between benign and malicious network behaviors. In this paper, we propose a novel end-to-end intrusion detection framework based on a contrastive learning approach. We design a hierarchical Convolutional Neural Network (CNN) and Gated Recurrent Unit (GRU) model to facilitate the automated extraction of spatiotemporal features from raw traffic data. The integration of contrastive learning amplifies the distinction between benign and malicious network traffic in the representation space. The proposed method exhibits enhanced detection capabilities for unknown attacks in comparison to the approaches trained using the cross-entropy loss function. Experiments are carried out on the public datasets CIC-IDS2017 and CSE-CIC-IDS2018, demonstrating that our method can attain a detection accuracy of 99.9% for known attacks, thus achieving state-of-the-art performance. For unknown attacks, a weighted recall rate of 95% can be achieved.

Industrial Product Surface Anomaly Detection with Realistic Synthetic Anomalies Based on Defect Map Prediction

The occurrence of anomalies on the surface of industrial products can lead to issues such as decreased product quality, reduced production efficiency, and safety hazards. Early detection and resolution of these problems are crucial for ensuring the quality and efficiency of production. The key challenge in applying deep learning to surface defect detection of industrial products is the scarcity of defect samples, which will make supervised learning methods unsuitable for surface defect detection problems. Therefore, it is a reasonable solution to use anomaly detection methods to deal with surface defect detection. Among image-based anomaly detection, reconstruction-based methods are the most commonly used. However, reconstruction-based approaches lack the involvement of defect samples in the training process, posing the risk of a perfect reconstruction of defects by the reconstruction network. In this paper, we propose a reconstruction-based defect detection algorithm that addresses these challenges by utilizing more realistic synthetic anomalies for training. Our model focuses on creating authentic synthetic defects and introduces an auto-encoder image reconstruction network with deep feature consistency constraints, as well as a defect separation network with a large receptive field. We conducted experiments on the challenging MVTec anomaly detection dataset and our trained model achieved an AUROC score of 99.70% and an average precision (AP) score of 99.87%. Our method surpasses recently proposed defect detection algorithms, thereby enhancing the accuracy of surface defect detection in industrial products.

Generating ICS Anomaly Data Reflecting Cyber-Attack Based on Systematic Sampling and Linear Regression

Cyber threats to industrial control systems (ICSs) have increased as information and communications technology (ICT) has been incorporated. In response to these cyber threats, we are implementing a range of security equipment and specialized training programs. Anomaly data stemming from cyber-attacks are crucial for effectively testing security equipment and conducting cyber training exercises. However, securing anomaly data in an ICS environment requires a lot of effort. For this reason, we propose a method for generating anomaly data that reflects cyber-attack characteristics. This method uses systematic sampling and linear regression models in an ICS environment to generate anomaly data reflecting cyber-attack characteristics based on benign data. The method uses statistical analysis to identify features indicative of cyber-attack characteristics and alters their values from benign data through systematic sampling. The transformed data are then used to train a linear regression model. The linear regression model can predict features because it has learned the linear relationships between data features. This experiment used ICS_PCAPS data generated based on Modbus, frequently used in ICS. In this experiment, more than 50,000 new anomaly data pieces were generated. As a result of using some of the new anomaly data generated as training data for the existing model, no significant performance degradation occurred. Additionally, comparing some of the new anomaly data with the original benign and attack data using kernel density estimation confirmed that the new anomaly data pattern was changing from benign data to attack data. In this way, anomaly data that partially reflect the pattern of the attack data were created. The proposed method generates anomaly data like cyber-attack data quickly and logically, free from the constraints of cost, time, and original cyber-attack data required in existing research.

Support vector data description with kernel density estimation (SVDD-KDE) control chart for network intrusion monitoring

Multivariate control charts have been applied in many sectors. One of the sectors that employ this method is network intrusion detection. However, the issue arises when the conventional control chart faces difficulty monitoring the network-traffic data that do not follow a normal distribution as required. Consequently, more false alarms will be found when inspecting network traffic data. To settle this problem, support vector data description (SVDD) is suggested. The control chart based on the SVDD distance can be applied for the non-normal distribution, even the unknown distributions. Kernel density estimation (KDE) is the nonparametric approach that can be applied in estimating the control limit of the non-parametric control charts. Based on these facts, a multivariate chart based on the integrated SVDD and KDE (SVDD-KDE) is proposed to monitor the network's anomaly. Simulation using the synthetic dataset is performed to examine the performance of the SVDD-KDE chart in detecting multivariate data shifts and outliers. Based on the simulation results, the proposed method produces better performance in detecting shifts and higher accuracy in detecting outliers. Further, the proposed method is applied in the intrusion detection system (IDS) to monitor network attacks. The NSL-KDD data is analyzed as the benchmark dataset. A comparison between the SVDD-KDE chart with the other IDS-based-control chart and the machine learning algorithms is executed. Although the it has high computational cost, the results show that the IDS based on the SVDD-KDE chart produces a high accuracy at 0.917 and AUC at 0.915 with a low false positive rate compared to several algorithms.

HH-NIDS: Heterogeneous Hardware-Based Network Intrusion Detection Framework for IoT Security

This study proposes a heterogeneous hardware-based framework for network intrusion detection using lightweight artificial neural network models. With the increase in the volume of exchanged data, IoT networks’ security has become a crucial issue. Anomaly-based intrusion detection systems (IDS) using machine learning have recently gained increased popularity due to their generation’s ability to detect unseen attacks. However, the deployment of anomaly-based AI-assisted IDS for IoT devices is computationally expensive. A high-performance and ultra-low power consumption anomaly-based IDS framework is proposed and evaluated in this paper. The framework has achieved the highest accuracy of 98.57% and 99.66% on the UNSW-NB15 and IoT-23 datasets, respectively. The inference engine on the MAX78000EVKIT AI-microcontroller is 11.3 times faster than the Intel Core i7-9750H 2.6 GHz and 21.3 times faster than NVIDIA GeForce GTX 1650 graphics cards, when the power drawn was 18mW. In addition, the pipelined design on the PYNQ-Z2 SoC FPGA board with the Xilinx Zynq xc7z020-1clg400c device is optimised to run at the on-chip frequency (100 MHz), which shows a speedup of 53.5 times compared to the MAX78000EVKIT.

An Intrusion Detection Method for Industrial Control System Based on Machine Learning

The integration of communication networks and the internet of industrial control in Industrial Control System (ICS) increases their vulnerability to cyber attacks, causing devastating outcomes. Traditional Intrusion Detection Systems (IDS) largely rely on predefined models and are trained mostly on specific cyber attacks, which means the traditional IDS cannot cope with unknown attacks. Additionally, most IDS do not consider the imbalanced nature of ICS datasets, thus suffering from low accuracy and high False Positive Rates when being put to use. In this paper, we propose the NCO–double-layer DIFF_RF–OPFYTHON intrusion detection method for ICS, which consists of NCO modules, double-layer DIFF_RF modules, and OPFYTHON modules. Detected traffic will be divided into three categories by the double-layer DIFF_RF module: known attacks, unknown attacks, and normal traffic. Then, the known attacks will be classified into specific attacks by the OPFYTHON module according to the feature of attack traffic. Finally, we use the NCO module to improve the model input and enhance the accuracy of the model. The results show that the proposed method outperforms traditional intrusion detection methods, such as XGboost and SVM. The detection of unknown attacks is also considerable. The accuracy of the dataset used in this paper reaches 98.13%. The detection rates for unknown attacks and known attacks reach 98.21% and 95.1%, respectively. Moreover, the method we proposed has achieved suitable results on other public datasets.

Detection and quantification of anomalies in communication networks based on LSTM-ARIMA combined model

The anomaly detection for communication networks is significant for improve the quality of communication services and network reliability. However, traditional communication monitoring methods lack proactive monitoring and real-time alerts and the prediction effect of a single machine learning model on communication data containing multiple features is not ideal. To solve the problem, A prediction-then-detection anomaly detection method was proposed, and quantitative assessment of network anomalies was developed. Specifically, anomaly-free data was obtained by eliminating outliers, and the long short-term memory (LSTM) and autoregressive integral moving average (ARIMA) were combined via residual weighting to predict the future state of the key performance indicators (KPI) without outliers. Anomalies were identified using the error comparison between the prediction and actual values, and the network condition was quantified using the scoring method. It is observed that the proposed LSTM-ARIMA hybrid model has better prediction effect, which can well represent the performance of KPIs of the future state, and the prediction-then-detection anomaly detection method has excellent performance on both precision and recall.