Respite for SMEs: A Systematic Review of Socio-Technical Cybersecurity Metrics

Performance of active learning models for screening prioritization in systematic reviews: a simulation study into the Average Time to Discover relevant records

BACKGROUND

Conducting a systematic review demands a significant amount of effort in screening titles and abstracts. To accelerate this process, various tools that utilize active learning have been proposed. These tools allow the reviewer to interact with machine learning software to identify relevant publications as early as possible. The goal of this study is to gain a comprehensive understanding of active learning models for reducing the workload in systematic reviews through a simulation study.

METHODS

The simulation study mimics the process of a human reviewer screening records while interacting with an active learning model. Different active learning models were compared based on four classification techniques (naive Bayes, logistic regression, support vector machines, and random forest) and two feature extraction strategies (TF-IDF and doc2vec). The performance of the models was compared for six systematic review datasets from different research areas. The evaluation of the models was based on the Work Saved over Sampling (WSS) and recall. Additionally, this study introduces two new statistics, Time to Discovery (TD) and Average Time to Discovery (ATD).

RESULTS

The models reduce the number of publications needed to screen by 91.7 to 63.9% while still finding 95% of all relevant records (WSS@95). Recall of the models was defined as the proportion of relevant records found after screening 10% of of all records and ranges from 53.6 to 99.8%. The ATD values range from 1.4% till 11.7%, which indicate the average proportion of labeling decisions the researcher needs to make to detect a relevant record. The ATD values display a similar ranking across the simulations as the recall and WSS values.

CONCLUSIONS

Active learning models for screening prioritization demonstrate significant potential for reducing the workload in systematic reviews. The Naive Bayes + TF-IDF model yielded the best results overall. The Average Time to Discovery (ATD) measures performance of active learning models throughout the entire screening process without the need for an arbitrary cut-off point. This makes the ATD a promising metric for comparing the performance of different models across different datasets.

Extending the Frontiers of Electronic Commerce Knowledge through Cybersecurity

As technology becomes more sophisticated so do cyber-attacks. The resilience of electronic commerce organisations represents a critical point nowadays because it influences consumer and digital business behaviour. In this context, the cybersecurity and electronic commerce knowledge were reviewed as a unit. The main aim of this paper is to support researchers and managers in understanding the theoretical framework and to provide a knowledge-based model. To achieve this aim, the authors performed an analysis of 14,585 papers from the Web of Science Core Collection that generated two visualized networks, analyzed with the metrics mean silhouette, modularity, betweenness centrality, and citation bursts in the context of digital resilience. The mapping process results show that the human factor represents the central junction with the fear of cyber-attacks and the perception of online shopping as risky. The adoption of electronic commerce and mobile commerce are two challenging research lines in the global economic resilience. Their adoption enabled by big data, artificial intelligence, machine learning, and even blockchain technology can strengthen resilience even when cybersecurity education is needed.

A Shared Cyber Threat Intelligence Solution for SMEs

Small- and medium-sized enterprises (SMEs) frequently experience cyberattacks, but often do not have the means to counter these attacks. Therefore, cybersecurity researchers and practitioners need to aid SMEs in their defence against cyber threats. Research has shown that SMEs require solutions that are automated and adapted to their context. In recent years, we have seen a surge in initiatives to share cyber threat intelligence (CTI) to improve collective cybersecurity resilience. Shared CTI has the potential to answer the SME call for automated and adaptable solutions. Sadly, as we demonstrate in this paper, current shared intelligence approaches scarcely address SME needs. We must investigate how shared CTI can be used to improve SME cybersecurity resilience. In this paper, we tackle this challenge using a systematic review to discover current state-of-the-art approaches to using shared CTI. We find that threat intelligence sharing platforms such as MISP have the potential to address SME needs, provided that the shared intelligence is turned into actionable insights. Based on this observation, we developed a prototype application that processes MISP data automatically, prioritises cybersecurity threats for SMEs, and provides SMEs with actionable recommendations tailored to their context. Subsequent evaluations in operational environments will help to improve our application, such that SMEs are enabled to thwart cyberattacks in future.

Machine Learning Cybersecurity Adoption in Small and Medium Enterprises in Developed Countries

In many developed countries, the usage of artificial intelligence (AI) and machine learning (ML) has become important in paving the future path in how data is managed and secured in the small and medium enterprises (SMEs) sector. SMEs in these developed countries have created their own cyber regimes around AI and ML. This knowledge is tested daily in how these countries’ SMEs run their businesses and identify threats and attacks, based on the support structure of the individual country. Based on recent changes to the UK General Data Protection Regulation (GDPR), Brexit, and ISO standards requirements, machine learning cybersecurity (MLCS) adoption in the UK SME market has become prevalent and a good example to lean on, amongst other developed nations. Whilst MLCS has been successfully applied in many applications, including network intrusion detection systems (NIDs) worldwide, there is still a gap in the rate of adoption of MLCS techniques for UK SMEs. Other developed countries such as Spain and Australia also fall into this category, and similarities and differences to MLCS adoptions are discussed. Applications of how MLCS is applied within these SME industries are also explored. The paper investigates, using quantitative and qualitative methods, the challenges to adopting MLCS in the SME ecosystem, and how operations are managed to promote business growth. Much like security guards and policing in the real world, the virtual world is now calling on MLCS techniques to be embedded like secret service covert operations to protect data being distributed by the millions into cyberspace. This paper will use existing global research from multiple disciplines to identify gaps and opportunities for UK SME small business cyber security. This paper will also highlight barriers and reasons for low adoption rates of MLCS in SMEs and compare success stories of larger companies implementing MLCS. The methodology uses structured quantitative and qualitative survey questionnaires, distributed across an extensive participation pool directed to the SMEs’ management and technical and non-technical professionals using stratify methods. Based on the analysis and findings, this study reveals that from the primary data obtained, SMEs have the appropriate cybersecurity packages in place but are not fully aware of their potential. Secondary data collection was run in parallel to better understand how these barriers and challenges emerged, and why the rate of adoption of MLCS was very low. The paper draws the conclusion that help through government policies and processes coupled together with collaboration could minimize cyber threats in combatting hackers and malicious actors in trying to stay ahead of the game. These aspirations can be reached by ensuring that those involved have been well trained and understand the importance of communication when applying appropriate safety processes and procedures. This paper also highlights important funding gaps that could help raise cyber security awareness in the form of grants, subsidies, and financial assistance through various public sector policies and training. Lastly, SMEs’ lack of understanding of risks and impacts of cybercrime could lead to conflicting messages between cross-company IT and cybersecurity rules. Trying to find the right balance between this risk and impact, versus productivity impact and costs, could lead to UK SMES getting over these hurdles in this cyberspace in the quest for promoting the usage of MLCS. UK and Wales governments can use the research conducted in this paper to inform and adapt their policies to help UK SMEs become more secure from cyber-attacks and compare them to other developed countries also on the same future path.