Big Data Analytics framework for Peer-to-Peer Botnet detection using Random Forests

Real-time botnet detection on large network bandwidths using machine learning

Botnets are one of the most harmful cyberthreats, that can perform many types of cyberattacks and cause billionaire losses to the global economy. Nowadays, vast amounts of network traffic are generated every second, hence manual analysis is impossible. To be effective, automatic botnet detection should be done as fast as possible, but carrying this out is difficult in large bandwidths. To handle this problem, we propose an approach that is capable of carrying out an ultra-fast network analysis (i.e. on windows of one second), without a significant loss in the F1-score. We compared our model with other three literature proposals, and achieved the best performance: an F1 score of 0.926 with a processing time of 0.007 ms per sample. We also assessed the robustness of our model on saturated networks and on large bandwidths. In particular, our model is capable of working on networks with a saturation of 10% of packet loss, and we estimated the number of CPU cores needed to analyze traffic on three bandwidth sizes. Our results suggest that using commercial-grade cores of 2.4 GHz, our approach would only need four cores for bandwidths of 100 Mbps and 1 Gbps, and 19 cores on 10 Gbps networks.

Using honeypots to model botnet attacks on the internet of medical things

Corona Virus Disease 2019 (COVID-19) has led to an increase in attacks targeting widespread smart devices. A vulnerable device can join multiple botnets simultaneously or sequentially. When different attack patterns are mixed with attack records, the security analyst produces an inaccurate report. There are numerous studies on botnet detection, but there is no publicly available solution to classify attack patterns based on the control periods. To fill this gap, we propose a novel data-driven method based on an intuitive hypothesis: bots tend to show time-related attack patterns within the same botnet control period. We deploy 462 honeypots in 22 countries to capture real-world attack activities and propose an algorithm to identify control periods. Experiments have demonstrated our method's efficacy. Besides, we present eight interesting findings that will help the security community better understand and fight botnet attacks now and in the future.

Design and Protection Strategy of Distributed Intrusion Detection System in Big Data Environment

One of the important research topics is protecting the host from threats by developing a reliable and accurate intrusion detection system. However, since the amount of data has grown fast due to the emergence of big data, the performance of traditional systems designed to identify breaches has suffered several flaws. One of them, for example, is known as single-point failure; low adaptability and a high false alarm rate are also typical. Hadoop is used to detect intrusions to tackle these difficulties. The Java system is used to create a framework with a significant data flow that detects intrusions when a distributed system is built. The proposed solution employs a distributed operating system for data collection, storage, and analysis. The results indicate that external distributed denial of service (DDoS) attacks are recognized quickly. The single-point failure issue is overcome, alleviating the bottleneck problem of data processing ability.

SUKRY: Suricata IDS with Enhanced kNN Algorithm on Raspberry Pi for Classifying IoT Botnet Attacks

The focus of this research is the application of the k-Nearest Neighbor algorithm in terms of classifying botnet attacks in the IoT environment. The kNN algorithm has several advantages in classification tasks, such as simplicity, effectiveness, and robustness. However, it does not perform well in handling large datasets such as the Bot-IoT dataset, which represents a huge amount of data about botnet attacks on IoT networks. Therefore, improving the kNN performance in classifying IoT botnet attacks is the main concern in this study by applying several feature selection techniques. The whole research process was conducted in the Rapidminer environment using three prebuilt feature selection techniques, namely, Information Gain, Forward Selection, and Backward Elimination. After comparing accuracy, precision, recall, F1 score and processing time, the combination of the kNN algorithm and the Forward Selection technique (kNN-FS) achieves the best results among others, with the highest level of accuracy and the fastest execution time among others. Finally, kNN-FS is used in developing SUKRY, which stands for Suricata IDS with Enhanced kNN Algorithm on Raspberry Pi.

Distributed Denial-of-Service (DDoS) Attacks and Defence Mechanisms in Various Web-enabled Computing Platforms

The demand for Internet security has escalated in the last two decades because the rapid proliferation in the number of Internet users has presented attackers with new detrimental opportunities. One of the simple yet powerful attack, lurking around the Internet today, is the Distributed Denial-of-Service (DDoS) attack. The expeditious surge in the collaborative environments, like IoT, cloud computing and SDN, have provided attackers with countless new avenues to benefit from the distributed nature of DDoS attacks. The attackers protect their anonymity by infecting distributed devices and utilizing them to create a bot army to constitute a large-scale attack. Thus, the development of an effective as well as efficient DDoS defense mechanism becomes an immediate goal. In this exposition, we present a DDoS threat analysis along with a few novel ground-breaking defense mechanisms proposed by various researchers for numerous domains. Further, we talk about popular performance metrics that evaluate the defense schemes. In the end, we list prevalent DDoS attack tools and open challenges.

An Ensemble of Prediction and Learning Mechanism for Improving Accuracy of Anomaly Detection in Network Intrusion Environments

The connectivity of our surrounding objects to the internet plays a tremendous role in our daily lives. Many network applications have been developed in every domain of life, including business, healthcare, smart homes, and smart cities, to name a few. As these network applications provide a wide range of services for large user groups, the network intruders are prone to developing intrusion skills for attack and malicious compliance. Therefore, safeguarding network applications and things connected to the internet has always been a point of interest for researchers. Many studies propose solutions for intrusion detection systems and intrusion prevention systems. Network communities have produced benchmark datasets available for researchers to improve the accuracy of intrusion detection systems. The scientific community has presented data mining and machine learning-based mechanisms to detect intrusion with high classification accuracy. This paper presents an intrusion detection system based on the ensemble of prediction and learning mechanisms to improve anomaly detection accuracy in a network intrusion environment. The learning mechanism is based on automated machine learning, and the prediction model is based on the Kalman filter. Performance analysis of the proposed intrusion detection system is evaluated using publicly available intrusion datasets UNSW-NB15 and CICIDS2017. The proposed model-based intrusion detection accuracy for the UNSW-NB15 dataset is 98.801 percent, and the CICIDS2017 dataset is 97.02 percent. The performance comparison results show that the proposed ensemble model-based intrusion detection significantly improves the intrusion detection accuracy.

A Review on Machine Learning Approaches for Network Malicious Behavior Detection in Emerging Technologies

Network anomaly detection systems (NADSs) play a significant role in every network defense system as they detect and prevent malicious activities. Therefore, this paper offers an exhaustive overview of different aspects of anomaly-based network intrusion detection systems (NIDSs). Additionally, contemporary malicious activities in network systems and the important properties of intrusion detection systems are discussed as well. The present survey explains important phases of NADSs, such as pre-processing, feature extraction and malicious behavior detection and recognition. In addition, with regard to the detection and recognition phase, recent machine learning approaches including supervised, unsupervised, new deep and ensemble learning techniques have been comprehensively discussed; moreover, some details about currently available benchmark datasets for training and evaluating machine learning techniques are provided by the researchers. In the end, potential challenges together with some future directions for machine learning-based NADSs are specified.

DeepDetect: Detection of Distributed Denial of Service Attacks Using Deep Learning

At the advent of advanced wireless technology and contemporary computing paradigms, Distributed Denial of Service (DDoS) attacks on Web-based services have not only increased exponentially in number, but also in the degree of sophistication; hence the need for detecting these attacks within the ocean of communication packets is extremely important. DDoS attacks were initially projected toward the network and transport layers. Over the years, attackers have shifted their offensive strategies toward the application layer. The application layer attacks are potentially more detrimental and stealthier because of the attack traffic and the benign traffic flows being indistinguishable. The distributed nature of these attacks is difficult to combat as they may affect tangible computing resources apart from network bandwidth consumption. In addition, smart devices connected to the Internet can be infected and used as botnets to launch DDoS attacks. In this paper, we propose a novel deep neural network-based detection mechanism that uses feed-forward back-propagation for accurately discovering multiple application layer DDoS attacks. The proposed neural network architecture can identify and use the most relevant high level features of packet flows with an accuracy of 98% on the state-of-the-art dataset containing various forms of DDoS attacks.

NTARC: A Data Model for the Systematic Review of Network Traffic Analysis Research

The increased interest in secure and reliable communications has turned the analysis of network traffic data into a predominant topic. A high number of research papers propose methods to classify traffic, detect anomalies, or identify attacks. Although the goals and methodologies are commonly similar, we lack initiatives to categorize the data, methods, and findings systematically. In this paper, we present Network Traffic Analysis Research Curation (NTARC), a data model to store key information about network traffic analysis research. We additionally use NTARC to perform a critical review of the field of research conducted in the last two decades. The collection of descriptive research summaries enables the easy retrieval of relevant information and a better reuse of past studies by the application of quantitative analysis. Among others benefits, it enables the critical review of methodologies, the detection of common flaws, the obtaining of baselines, and the consolidation of best practices. Furthermore, it provides a basis to achieve reproducibility, a key requirement that has long been undervalued in the area of traffic analysis. Thus, besides reading hard copies of papers, with NTARC, researchers can make use of a digital environment that facilitates queries and reviews over a comprehensive field corpus.

Classification Spam Email with Elimination of Unsuitable Features with Hybrid of GA-Naive Bayes

Email spam is a security problem that involves different techniques in machine learning to solve this problem. The rise of this security issue makes organisation email service unreliable and has a direct relation with vulnerability of clients through unexpected spam mails, like ransomware. There are several methods to identifying spam emails. Most of these methods focused on feature selection; however, these models decreased the accuracy of the detection. This paper proposed a novel spam detection method that is not only to decrease the accuracy, but eliminates unsuitable features with less processing. The features are in the terms of contents, and the number of features is very big, so it can decrease the memory complexity. We use Hewlett-Packet (HP) laboratory samples text emails. First, GA algorithm is employed to select features without limited number of feature selection with the aid of Bayesian theory as a fitness function and checked with a different number of repetitions. The result improved with GA by increasing number of repetitions, and tested with distinctive selection method, Random selection and Tournament selection. In the second stage, the dataset classifies emails as Spam or Ham by Naive Bayes. The results show that Naive Bayes and hybrid GA-Naive Bayes are almost identical, but GA-Naive Bayes has a better performance.

Wireless Sensor Networks Intrusion Detection Based on SMOTE and the Random Forest Algorithm

With the wide application of wireless sensor networks in military and environmental monitoring, security issues have become increasingly prominent. Data exchanged over wireless sensor networks is vulnerable to malicious attacks due to the lack of physical defense equipment. Therefore, corresponding schemes of intrusion detection are urgently needed to defend against such attacks. Considering the serious class imbalance of the intrusion dataset, this paper proposes a method of using the synthetic minority oversampling technique (SMOTE) to balance the dataset and then uses the random forest algorithm to train the classifier for intrusion detection. The simulations are conducted on a benchmark intrusion dataset, and the accuracy of the random forest algorithm has reached 92.39%, which is higher than other comparison algorithms. After oversampling the minority samples, the accuracy of the random forest combined with the SMOTE has increased to 92.57%. This shows that the proposed algorithm provides an effective solution to solve the problem of class imbalance and improves the performance of intrusion detection.

Using wearable sensors to classify subject-specific running biomechanical gait patterns based on changes in environmental weather conditions

Running-related overuse injuries can result from a combination of various intrinsic (e.g., gait biomechanics) and extrinsic (e.g., running surface) risk factors. However, it is unknown how changes in environmental weather conditions affect running gait biomechanical patterns since these data cannot be collected in a laboratory setting. Therefore, the purpose of this study was to develop a classification model based on subject-specific changes in biomechanical running patterns across two different environmental weather conditions using data obtained from wearable sensors in real-world environments. Running gait data were recorded during winter and spring sessions, with recorded average air temperatures of -10° C and +6° C, respectively. Classification was performed based on measurements of pelvic drop, ground contact time, braking, vertical oscillation of pelvis, pelvic rotation, and cadence obtained from 66,370 strides (~11,000/runner) from a group of recreational runners. A non-linear and ensemble machine learning algorithm, random forest (RF), was used to classify and compute a heuristic for determining the importance of each variable in the prediction model. To validate the developed subject-specific model, two cross-validation methods (one-against-another and partitioning datasets) were used to obtain experimental mean classification accuracies of 87.18% and 95.42%, respectively, indicating an excellent discriminatory ability of the RF-based model. Additionally, the ranked order of variable importance differed across the individual runners. The results from the RF-based machine-learning algorithm demonstrates that processing gait biomechanical signals from a single wearable sensor can successfully detect changes to an individual's running patterns based on data obtained in real-world environments.

An Insight into State-of-the-Art Techniques for Big Data Classification

This article describes how classification algorithms have emerged as strong meta-learning techniques to accurately and efficiently analyze the masses of data generated from the widespread use of internet and other sources. In particular, there is need of some mechanism which classifies unstructured data into some organized form. Classification techniques over big transactional database may provide required data to the users from large datasets in a more simplified way. With the intention of organizing and clearly representing the current state of classification algorithms for big data, present paper discusses various concepts and algorithms, and also an exhaustive review of existing classification algorithms over big data classification frameworks and other novel frameworks. The paper provides a comprehensive comparison, both from a theoretical as well as an empirical perspective. The effectiveness of the candidate classification algorithms is measured through a number of performance metrics such as implementation technique, data source validation, and scalability etc.

A survey of cloud-based network intrusion detection analysis

As network traffic grows and attacks become more prevalent and complex, we must find creative new ways to enhance intrusion detection systems (IDSes). Recently, researchers have begun to harness both machine learning and cloud computing technology to better identify threats and speed up computation times. This paper explores current research at the intersection of these two fields by examining cloud-based network intrusion detection approaches that utilize machine learning algorithms (MLAs). Specifically, we consider clustering and classification MLAs, their applicability to modern intrusion detection, and feature selection algorithms, in order to underline prominent implementations from recent research. We offer a current overview of this growing body of research, highlighting successes, challenges, and future directions for MLA-usage in cloud-based network intrusion detection approaches.

An efficient reversible privacy-preserving data mining technology over data streams

With the popularity of smart handheld devices and the emergence of cloud computing, users and companies can save various data, which may contain private data, to the cloud. Topics relating to data security have therefore received much attention. This study focuses on data stream environments and uses the concept of a sliding window to design a reversible privacy-preserving technology to process continuous data in real time, known as a continuous reversible privacy-preserving (CRP) algorithm. Data with CRP algorithm protection can be accurately recovered through a data recovery process. In addition, by using an embedded watermark, the integrity of the data can be verified. The results from the experiments show that, compared to existing algorithms, CRP is better at preserving knowledge and is more effective in terms of reducing information loss and privacy disclosure risk. In addition, it takes far less time for CRP to process continuous data than existing algorithms. As a result, CRP is confirmed as suitable for data stream environments and fulfills the requirements of being lightweight and energy-efficient for smart handheld devices.