An Entropy-Based Network Anomaly Detection Method

Efficient First-Order Algorithms for Large-Scale, Non-Smooth Maximum Entropy Models with Application to Wildfire Science

Maximum entropy (MaxEnt) models are a class of statistical models that use the maximum entropy principle to estimate probability distributions from data. Due to the size of modern data sets, MaxEnt models need efficient optimization algorithms to scale well for big data applications. State-of-the-art algorithms for MaxEnt models, however, were not originally designed to handle big data sets; these algorithms either rely on technical devices that may yield unreliable numerical results, scale poorly, or require smoothness assumptions that many practical MaxEnt models lack. In this paper, we present novel optimization algorithms that overcome the shortcomings of state-of-the-art algorithms for training large-scale, non-smooth MaxEnt models. Our proposed first-order algorithms leverage the Kullback-Leibler divergence to train large-scale and non-smooth MaxEnt models efficiently. For MaxEnt models with discrete probability distribution of n elements built from samples, each containing m features, the stepsize parameter estimation and iterations in our algorithms scale on the order of O(mn) operations and can be trivially parallelized. Moreover, the strong ℓ1 convexity of the Kullback-Leibler divergence allows for larger stepsize parameters, thereby speeding up the convergence rate of our algorithms. To illustrate the efficiency of our novel algorithms, we consider the problem of estimating probabilities of fire occurrences as a function of ecological features in the Western US MTBS-Interagency wildfire data set. Our numerical results show that our algorithms outperform the state of the art by one order of magnitude and yield results that agree with physical models of wildfire occurrence and previous statistical analyses of wildfire drivers.

Comparison of Information Criteria for Detection of Useful Signals in Noisy Environments

This paper considers the appearance of indications of useful acoustic signals in the signal/noise mixture. Various information characteristics (information entropy, Jensen-Shannon divergence, spectral information divergence and statistical complexity) are investigated in the context of solving this problem. Both time and frequency domains are studied for the calculation of information entropy. The effectiveness of statistical complexity is shown in comparison with other information metrics for different signal-to-noise ratios. Two different approaches for statistical complexity calculations are also compared. In addition, analytical formulas for complexity and disequilibrium are obtained using entropy variation in the case of signal spectral distribution. The connection between the statistical complexity criterion and the Neyman-Pearson approach for hypothesis testing is discussed. The effectiveness of the proposed approach is shown for different types of acoustic signals and noise models, including colored noises, and different signal-to-noise ratios, especially when the estimation of additional noise characteristics is impossible.

Towards More Efficient Rényi Entropy Estimation

Estimation of Rényi entropy is of fundamental importance to many applications in cryptography, statistical inference, and machine learning. This paper aims to improve the existing estimators with regard to: (a) the sample size, (b) the estimator adaptiveness, and (c) the simplicity of the analyses. The contribution is a novel analysis of the generalized "birthday paradox" collision estimator. The analysis is simpler than in prior works, gives clear formulas, and strengthens existing bounds. The improved bounds are used to develop an adaptive estimation technique that outperforms previous methods, particularly in regimes of low or moderate entropy. Last but not least, to demonstrate that the developed techniques are of broader interest, a number of applications concerning theoretical and practical properties of "birthday estimators" are discussed.

Two-Stage Intelligent Model for Detecting Malicious DDoS Behavior

5G technologies provide ubiquitous connectivity. However, 5G security is a particularly important issue. Moreover, because public datasets are outdated, we need to create a self-generated dataset on the virtual platform. Therefore, we propose a two-stage intelligent detection model to enable 5G networks to withstand security issues and threats. Finally, we define malicious traffic detection capability metrics. We apply the self-generated dataset and metrics to thoroughly evaluate the proposed mechanism. We compare our proposed method with benchmark statistics and neural network algorithms. The experimental results show that the two-stage intelligent detection model can distinguish between benign and abnormal traffic and classify 21 kinds of DDoS. Our analysis also shows that the proposed approach outperforms all the compared approaches in terms of detection rate, malicious traffic detection capability, and response time.

Deceptive Techniques to Hide a Compressed Video Stream for Information Security

With the recent development of video compression methods, video transmission on traditional devices and video distribution using networks has increased in various devices such as drones, IP cameras, and small IoT devices. As a result, the demand for encryption techniques such as MPEG-DASH for transmitting streams over networks is increasing. These video stream security methods guarantee stream confidentiality. However, they do not hide the fact that the encrypted stream is being transmitted over the network. Considering that sniffing attacks can analyze the entropy of the stream and scan huge amounts of traffic on the network, to solve this problem, the deception method is required, which appears unencrypted but a confidential stream. In this paper, we propose the new deception method that utilizes standard NAL unit rules of video codec, where the unpromised device shows the cover video and the promised device shows the secret video for deceptive security. This method allows a low encryption cost and the stream to dodge entropy-based sniffing scan attacks. The proposed stream shows that successful decoding using five standard decoders and processing performance was 61% faster than the conventional encryption method in the test signal conformance set. In addition, a network encrypted stream scan method the HEDGE showed classification results that our stream is similar to a compressed video.

Modeling and Analysis of Anomalies in the Network Infrastructure Based on the Potts Model

The paper discusses issues concerning the occurrence of anomalies affecting the process of phase transitions. The considered issue was examined from the perspective of phase transitions in network structures, particularly in IT networks, Internet of Things and Internet of Everything. The basis for the research was the Potts model in the context of IT networks. The author proposed the classification of anomalies in relation to the states of particular nodes in the network structure. Considered anomalies included homogeneous, heterogeneous, individual and cyclic disorders. The results of tests and simulations clearly showed the impact of anomalies on the phase transitions in the network structures. The obtained results can be applied in modelling the processes occurring in network structures, particularly in IT networks.

Benchmarking Analysis of the Accuracy of Classification Methods Related to Entropy

In the machine learning literature we can find numerous methods to solve classification problems. We propose two new performance measures to analyze such methods. These measures are defined by using the concept of proportional reduction of classification error with respect to three benchmark classifiers, the random and two intuitive classifiers which are based on how a non-expert person could realize classification simply by applying a frequentist approach. We show that these three simple methods are closely related to different aspects of the entropy of the dataset. Therefore, these measures account somewhat for entropy in the dataset when evaluating the performance of classifiers. This allows us to measure the improvement in the classification results compared to simple methods, and at the same time how entropy affects classification capacity. To illustrate how these new performance measures can be used to analyze classifiers taking into account the entropy of the dataset, we carry out an intensive experiment in which we use the well-known J48 algorithm, and a UCI repository dataset on which we have previously selected a subset of the most relevant attributes. Then we carry out an extensive experiment in which we consider four heuristic classifiers, and 11 datasets.

Microcanonical and Canonical Ensembles for fMRI Brain Networks in Alzheimer's Disease

This paper seeks to advance the state-of-the-art in analysing fMRI data to detect onset of Alzheimer's disease and identify stages in the disease progression. We employ methods of network neuroscience to represent correlation across fMRI data arrays, and introduce novel techniques for network construction and analysis. In network construction, we vary thresholds in establishing BOLD time series correlation between nodes, yielding variations in topological and other network characteristics. For network analysis, we employ methods developed for modelling statistical ensembles of virtual particles in thermal systems. The microcanonical ensemble and the canonical ensemble are analogous to two different fMRI network representations. In the former case, there is zero variance in the number of edges in each network, while in the latter case the set of networks have a variance in the number of edges. Ensemble methods describe the macroscopic properties of a network by considering the underlying microscopic characterisations which are in turn closely related to the degree configuration and network entropy. When applied to fMRI data in populations of Alzheimer's patients and controls, our methods demonstrated levels of sensitivity adequate for clinical purposes in both identifying brain regions undergoing pathological changes and in revealing the dynamics of such changes.

Network Traffic Classification by Program Synthesis

Writing classification rules to identify interesting network traffic is a time-consuming and error-prone task. Learning-based classification systems automatically extract such rules from positive and negative traffic examples. However, due to limitations in the representation of network traffic and the learning strategy, these systems lack both expressiveness to cover a range of applications and interpretability in fully describing the traffic’s structure at the session layer. This paper presents Sharingan system, which uses program synthesis techniques to generate network classification programs at the session layer. Sharingan accepts raw network traces as inputs and reports potential patterns of the target traffic in NetQRE, a domain specific language designed for specifying session-layer quantitative properties. We develop a range of novel optimizations that reduce the synthesis time for large and complex tasks to a matter of minutes. Our experiments show that Sharingan is able to correctly identify patterns from a diverse set of network traces and generates explainable outputs, while achieving accuracy comparable to state-of-the-art learning-based systems.

On the Application of Entropy Measures with Sliding Window for Intrusion Detection in Automotive In-Vehicle Networks

The evolution of modern automobiles to higher levels of connectivity and automatism has also increased the need to focus on the mitigation of potential cybersecurity risks. Researchers have proven in recent years that attacks on in-vehicle networks of automotive vehicles are possible and the research community has investigated various cybersecurity mitigation techniques and intrusion detection systems which can be adopted in the automotive sector. In comparison to conventional intrusion detection systems in large fixed networks and ICT infrastructures in general, in-vehicle systems have limited computing capabilities and other constraints related to data transfer and the management of cryptographic systems. In addition, it is important that attacks are detected in a short time-frame as cybersecurity attacks in vehicles can lead to safety hazards. This paper proposes an approach for intrusion detection of cybersecurity attacks in in-vehicle networks, which takes in consideration the constraints listed above. The approach is based on the application of an information entropy-based method based on a sliding window, which is quite efficient from time point of view, it does not require the implementation of complex cryptographic systems and it still provides a very high detection accuracy. Different entropy measures are used in the evaluation: Shannon Entropy, Renyi Entropy, Sample Entropy, Approximate Entropy, Permutation Entropy, Dispersion and Fuzzy Entropy. This paper evaluates the impact of the different hyperparameters present in the definition of entropy measures on a very large public data set of CAN-bus traffic with millions of CAN-bus messages with four different types of attacks: Denial of Service, Fuzzy Attack and two spoofing attacks related to RPM and Gear information. The sliding window approach in combination with entropy measures can detect attacks in a time-efficient way and with great accuracy for specific choices of the hyperparameters and entropy measures.

Anomaly Detection for Individual Sequences with Applications in Identifying Malicious Tools

Anomaly detection refers to the problem of identifying abnormal behaviour within a set of measurements. In many cases, one has some statistical model for normal data, and wishes to identify whether new data fit the model or not. However, in others, while there are normal data to learn from, there is no statistical model for this data, and there is no structured parameter set to estimate. Thus, one is forced to assume an individual sequences setup, where there is no given model or any guarantee that such a model exists. In this work, we propose a universal anomaly detection algorithm for one-dimensional time series that is able to learn the normal behaviour of systems and alert for abnormalities, without assuming anything on the normal data, or anything on the anomalies. The suggested method utilizes new information measures that were derived from the Lempel–Ziv (LZ) compression algorithm in order to optimally and efficiently learn the normal behaviour (during learning), and then estimate the likelihood of new data (during operation) and classify it accordingly. We apply the algorithm to key problems in computer security, as well as a benchmark anomaly detection data set, all using simple, single-feature time-indexed data. The first is detecting Botnets Command and Control (C&C) channels without deep inspection. We then apply it to the problems of malicious tools detection via system calls monitoring and data leakage identification.We conclude with the New York City (NYC) taxi data. Finally, while using information theoretic tools, we show that an attacker’s attempt to maliciously fool the detection system by trying to generate normal data is bound to fail, either due to a high probability of error or because of the need for huge amounts of resources.

A Multiple Rényi Entropy Based Intrusion Detection System for Connected Vehicles

In this paper, we propose an intrusion detection system based on the estimation of the Rényi entropy with multiple orders. The Rényi entropy is a generalized notion of entropy that includes the Shannon entropy and the min-entropy as special cases. In 2018, Kim proposed an efficient estimation method for the Rényi entropy with an arbitrary real order α. In this work, we utilize this method to construct a multiple order, Rényi entropy based intrusion detection system (IDS) for vehicular systems with various network connections. The proposed method estimates the Rényi entropies simultaneously with three distinct orders, two, three, and four, based on the controller area network (CAN)-IDs of consecutively generated frames. The collected frames are split into blocks with a fixed number of frames, and the entropies are evaluated based on these blocks. For a more accurate estimation against each type of attack, we also propose a retrospective sliding window method for decision of attacks based on the estimated entropies. For fair comparison, we utilized the CAN-ID attack data set generated by a research team from Korea University. Our results show that the proposed method can show the false negative and positive errors of less than 1% simultaneously.

An Entropy-Based Car Failure Detection Method Based on Data Acquisition Pipeline

Modern cars are equipped with plenty of electronic devices called Electronic Control Units (ECU). ECUs collect diagnostic data from a car’s components such as the engine, brakes etc. These data are then processed, and the appropriate information is communicated to the driver. From the point of view of safety of the driver and the passengers, the information about the car faults is vital. Regardless of the development of on-board computers, only a small amount of information is passed on to the driver. With the data mining approach, it is possible to obtain much more information from the data than it is provided by standard car equipment. This paper describes the environment built by the authors for data collection from ECUs. The collected data have been processed using parameterized entropies and data mining algorithms. Finally, we built a classifier able to detect a malfunctioning thermostat even if the car equipment does not indicate it.

Learning Entropy as a Learning-Based Information Concept

Recently, a novel concept of a non-probabilistic novelty detection measure, based on a multi-scale quantification of unusually large learning efforts of machine learning systems, was introduced as learning entropy (LE). The key finding with LE is that the learning effort of learning systems is quantifiable as a novelty measure for each individually observed data point of otherwise complex dynamic systems, while the model accuracy is not a necessary requirement for novelty detection. This brief paper extends the explanation of LE from the point of an informatics approach towards a cognitive (learning-based) information measure emphasizing the distinction from Shannon’s concept of probabilistic information. Fundamental derivations of learning entropy and of its practical estimations are recalled and further extended. The potentials, limitations, and, thus, the current challenges of LE are discussed.

Cross-Method-Based Analysis and Classification of Malicious Behavior by API Calls Extraction

Data-driven public security networking and computer systems are always under threat from malicious codes known as malware; therefore, a large amount of research and development is taking place to find effective countermeasures. These countermeasures are mainly based on dynamic and statistical analysis. Because of the obfuscation techniques used by the malware authors, security researchers and the anti-virus industry are facing a colossal issue regarding the extraction of hidden payloads within packed executable extraction. Based on this understanding, we first propose a method to de-obfuscate and unpack the malware samples. Additional, cross-method-based big data analysis to dynamically and statistically extract features from malware has been proposed. The Application Programming Interface (API) call sequences that reflect the malware behavior of its code have been used to detect behavior such as network traffic, modifying a file, writing to stderr or stdout, modifying a registry value, creating a process. Furthermore, we include a similarity analysis and machine learning algorithms to profile and classify malware behaviors. The experimental results of the proposed method show that malware detection accuracy is very useful to discover potential threats and can help the decision-maker to deploy appropriate countermeasures.

Using Entropy in Web Usage Data Preprocessing

The paper is focused on an examination of the use of entropy in the field of web usage mining. Entropy creates an alternative possibility of determining the ratio of auxiliary pages in the session identification using the Reference Length method. The experiment was conducted on two different web portals. The first log file was obtained from a course of virtual learning environment web portal. The second log file was received from the web portal with anonymous access. A comparison of the results of entropy estimation of the ratio of auxiliary pages and a sitemap estimation of the ratio of auxiliary pages showed that in the case of sitemap abundance, entropy could be a full-valued substitution for the estimate of the ratio of auxiliary pages.