The Comparison of Cybersecurity Datasets

The use of artificial intelligence to detect voided medication orders in oral and maxillofacial surgery inpatients

OBJECTIVE

The aim of this study is to determine if supervised machine learning algorithms can accurately predict voided computerized physician order entry in oral and maxillofacial surgery inpatients.

METHODS

Data from Electronic Medical Record included patient demographics, comorbidities, procedures, vital signs, laboratory values, and medication orders were retrospectively collected. Predictor variables included patient demographics, comorbidities, procedures, vital signs, and laboratory values. Outcome of interest is if a medication order was voided or not. Data was cleaned and processed using Microsoft Excel and Python v3.12. Gradient Boosted Decision Trees, Random Forest, K-Nearest Neighbor, and Naïve Bayes were trained, validated, and tested for accuracy of the prediction of voided medication orders.

RESULTS

37,493 medication orders from 1,204 patient admissions over 5 years were used for this study. 3,892 (10.4%) medication orders were voided. Gradient Boosted Decision Trees, Random Forest, K-Nearest Neighbor, and Naïve Bayes had an Area Under the Receiver Operating Curve of 0.802 with 95% CI [0.787, 0.825], 0.746 with 95% CI [0.722, 0.765], 0.685 with 95% CI [0.667, 0.699], and 0.505 with 95% CI [0.489, 0.539], respectively. Area Under the Precision Recall Curve was 0.684 with 95% CI [0.679, 0.702], 0.647 with 95% CI [0.638, 0.664], 0.429 with 95% CI [0.417, 0.434], and 0.551 with 95% CI [0.551, 0.552], respectively.

CONCLUSION

Gradient Boosted Decision Trees was the best performing model of the supervised machine learning algorithms with satisfactory outcomes in the test cohort for predicting voided Computerized Physician Order Entry in Oral and Maxillofacial Surgery inpatients.

ReinforSec: An Automatic Generator of Synthetic Malware Samples and Denial-of-Service Attacks through Reinforcement Learning

In recent years, cybersecurity has been strengthened through the adoption of processes, mechanisms and rapid sources of indicators of compromise in critical areas. Among the most latent challenges are the detection, classification and eradication of malware and Denial of Service Cyber-Attacks (DoS). The literature has presented different ways to obtain and evaluate malware- and DoS-cyber-attack-related instances, either from a technical point of view or by offering ready-to-use datasets. However, acquiring fresh, up-to-date samples requires an arduous process of exploration, sandbox configuration and mass storage, which may ultimately result in an unbalanced or under-represented set. Synthetic sample generation has shown that the cost associated with setting up controlled environments and time spent on sample evaluation can be reduced. Nevertheless, the process is performed when the observations already belong to a characterized set, totally detached from a real environment. In order to solve the aforementioned, this work proposes a methodology for the generation of synthetic samples of malicious Portable Executable binaries and DoS cyber-attacks. The task is performed via a Reinforcement Learning engine, which learns from a baseline of different malware families and DoS cyber-attack network properties, resulting in new, mutated and highly functional samples. Experimental results demonstrate the high adaptability of the outputs as new input datasets for different Machine Learning algorithms.

Introducing the CYSAS-S3 Dataset for Operationalizing a Mission-Oriented Cyber Situational Awareness

The digital transformation of the defence sector is not exempt from innovative requirements and challenges, with the lack of availability of reliable, unbiased and consistent data for training automatisms (machine learning algorithms, decision-making, what-if recreation of operational conditions, support the human understanding of the hybrid operational picture, personnel training/education, etc.) being one of the most relevant gaps. In the context of cyber defence, the state-of-the-art provides a plethora of data network collections that tend to lack presenting the information of all communication layers (physical to application). They are synthetically generated in scenarios far from the singularities of cyber defence operations. None of these data network collections took into consideration usage profiles and specific environments directly related to acquiring a cyber situational awareness, typically missing the relationship between incidents registered at the hardware/software level and their impact on the military mission assets and objectives, which consequently bypasses the entire chain of dependencies between strategic, operational, tactical and technical domains. In order to contribute to the mitigation of these gaps, this paper introduces CYSAS-S3, a novel dataset designed and created as a result of a joint research action that explores the principal needs for datasets by cyber defence centres, resulting in the generation of a collection of samples that correlate the impact of selected Advanced Persistent Threats (APT) with each phase of their cyber kill chain, regarding mission-level operations and goals.

Measurements of User and Sensor Data from the Internet of Things (IoT) Devices

The evolution of modern cyber-physical systems and the tremendous growth in the number of interconnected Internet of Things (IoT) devices are already paving new ways for the development of improved data collection and processing methods [...]