1
|
Zhao M, Peng H, Li L, Ren Y. Graph Attention Network and Informer for Multivariate Time Series Anomaly Detection. SENSORS (BASEL, SWITZERLAND) 2024; 24:1522. [PMID: 38475058 DOI: 10.3390/s24051522] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/12/2024] [Revised: 02/16/2024] [Accepted: 02/23/2024] [Indexed: 03/14/2024]
Abstract
Time series anomaly detection is very important to ensure the security of industrial control systems (ICSs). Many algorithms have performed well in anomaly detection. However, the performance of most of these algorithms decreases sharply with the increase in feature dimension. This paper proposes an anomaly detection scheme based on Graph Attention Network (GAT) and Informer. GAT learns sequential characteristics effectively, and Informer performs excellently in long time series prediction. In addition, long-time forecasting loss and short-time forecasting loss are used to detect multivariate time series anomalies. Short-time forecasting is used to predict the next time value, and long-time forecasting is employed to assist the short-time prediction. We conduct a large number of experiments on industrial control system datasets SWaT and WADI. Compared with most advanced methods, we achieve competitive results, especially on higher-dimensional datasets. Moreover, the proposed method can accurately locate anomalies and realize interpretability.
Collapse
|
2
|
Lee JH, Ji IH, Jeon SH, Seo JT. Generating ICS Anomaly Data Reflecting Cyber-Attack Based on Systematic Sampling and Linear Regression. SENSORS (BASEL, SWITZERLAND) 2023; 23:9855. [PMID: 38139701 PMCID: PMC10747890 DOI: 10.3390/s23249855] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Revised: 12/04/2023] [Accepted: 12/11/2023] [Indexed: 12/24/2023]
Abstract
Cyber threats to industrial control systems (ICSs) have increased as information and communications technology (ICT) has been incorporated. In response to these cyber threats, we are implementing a range of security equipment and specialized training programs. Anomaly data stemming from cyber-attacks are crucial for effectively testing security equipment and conducting cyber training exercises. However, securing anomaly data in an ICS environment requires a lot of effort. For this reason, we propose a method for generating anomaly data that reflects cyber-attack characteristics. This method uses systematic sampling and linear regression models in an ICS environment to generate anomaly data reflecting cyber-attack characteristics based on benign data. The method uses statistical analysis to identify features indicative of cyber-attack characteristics and alters their values from benign data through systematic sampling. The transformed data are then used to train a linear regression model. The linear regression model can predict features because it has learned the linear relationships between data features. This experiment used ICS_PCAPS data generated based on Modbus, frequently used in ICS. In this experiment, more than 50,000 new anomaly data pieces were generated. As a result of using some of the new anomaly data generated as training data for the existing model, no significant performance degradation occurred. Additionally, comparing some of the new anomaly data with the original benign and attack data using kernel density estimation confirmed that the new anomaly data pattern was changing from benign data to attack data. In this way, anomaly data that partially reflect the pattern of the attack data were created. The proposed method generates anomaly data like cyber-attack data quickly and logically, free from the constraints of cost, time, and original cyber-attack data required in existing research.
Collapse
|
3
|
Nankya M, Chataut R, Akl R. Securing Industrial Control Systems: Components, Cyber Threats, and Machine Learning-Driven Defense Strategies. SENSORS (BASEL, SWITZERLAND) 2023; 23:8840. [PMID: 37960539 PMCID: PMC10649322 DOI: 10.3390/s23218840] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Revised: 10/27/2023] [Accepted: 10/27/2023] [Indexed: 11/15/2023]
Abstract
Industrial Control Systems (ICS), which include Supervisory Control and Data Acquisition (SCADA) systems, Distributed Control Systems (DCS), and Programmable Logic Controllers (PLC), play a crucial role in managing and regulating industrial processes. However, ensuring the security of these systems is of utmost importance due to the potentially severe consequences of cyber attacks. This article presents an overview of ICS security, covering its components, protocols, industrial applications, and performance aspects. It also highlights the typical threats and vulnerabilities faced by these systems. Moreover, the article identifies key factors that influence the design decisions concerning control, communication, reliability, and redundancy properties of ICS, as these are critical in determining the security needs of the system. The article outlines existing security countermeasures, including network segmentation, access control, patch management, and security monitoring. Furthermore, the article explores the integration of machine learning techniques to enhance the cybersecurity of ICS. Machine learning offers several advantages, such as anomaly detection, threat intelligence analysis, and predictive maintenance. However, combining machine learning with other security measures is essential to establish a comprehensive defense strategy for ICS. The article also addresses the challenges associated with existing measures and provides recommendations for improving ICS security. This paper becomes a valuable reference for researchers aiming to make meaningful contributions within the constantly evolving ICS domain by providing an in-depth examination of the present state, challenges, and potential future advancements.
Collapse
|
4
|
Ren Y, Feng K, Hu F, Chen L, Chen Y. A Lightweight Unsupervised Intrusion Detection Model Based on Variational Auto-Encoder. SENSORS (BASEL, SWITZERLAND) 2023; 23:8407. [PMID: 37896500 PMCID: PMC10611103 DOI: 10.3390/s23208407] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/27/2023] [Revised: 10/09/2023] [Accepted: 10/10/2023] [Indexed: 10/29/2023]
Abstract
With the gradual integration of internet technology and the industrial control field, industrial control systems (ICSs) have begun to access public networks on a large scale. Attackers use these public network interfaces to launch frequent invasions of industrial control systems, thus resulting in equipment failure and downtime, production data leakage, and other serious harm. To ensure security, ICSs urgently need a mature intrusion detection mechanism. Most of the existing research on intrusion detection in ICSs focuses on improving the accuracy of intrusion detection, thereby ignoring the problem of limited equipment resources in industrial control environments, which makes it difficult to apply excellent intrusion detection algorithms in practice. In this study, we first use the spectral residual (SR) algorithm to process the data; we then propose the improved lightweight variational autoencoder (LVA) with autoregression to reconstruct the data, and we finally perform anomaly determination based on the permutation entropy (PE) algorithm. We construct a lightweight unsupervised intrusion detection model named LVA-SP. The model as a whole adopts a lightweight design with a simpler network structure and fewer parameters, which achieves a balance between the detection accuracy and the system resource overhead. Experimental results on the ICSs dataset show that our proposed LVA-SP model achieved an F1-score of 84.81% and has advantages in terms of time and memory overhead.
Collapse
|
5
|
Borcherding A, Morawetz M, Pfrang S. Smarter Evolution: Enhancing Evolutionary Black Box Fuzzing with Adaptive Models. SENSORS (BASEL, SWITZERLAND) 2023; 23:7864. [PMID: 37765921 PMCID: PMC10537775 DOI: 10.3390/s23187864] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/04/2023] [Revised: 09/05/2023] [Accepted: 09/06/2023] [Indexed: 09/29/2023]
Abstract
Smart production ecosystems are a valuable target for attackers. In particular, due to the high level of connectivity introduced by Industry 4.0, attackers can potentially attack individual components of production systems from the outside. One approach to strengthening the security of industrial control systems is to perform black box security tests such as network fuzzing. These are applicable, even if no information on the internals of the control system is available. However, most security testing strategies assume a gray box setting, in which some information on the internals are available. We propose a new approach to bridge the gap between these gray box strategies and the real-world black box setting in the domain of industrial control systems. This approach involves training an adaptive machine learning model that approximates the information that is missing in a black box setting. We propose three different approaches for the model, combine them with an evolutionary testing approach, and perform an evaluation using a System under Test with known vulnerabilities. Our evaluation shows that the model is indeed able to learn valuable information about a previously unknown system, and that more vulnerabilities can be uncovered with our approach. The model-based approach using a Decision Tree was able to find a significantly higher number of vulnerabilities than the two baseline fuzzers.
Collapse
|
6
|
Zarzycki K, Chaber P, Cabaj K, Ławryńczuk M, Marusak P, Nebeluk R, Plamowski S, Wojtulewicz A. Forgery Cyber-Attack Supported by LSTM Neural Network: An Experimental Case Study. SENSORS (BASEL, SWITZERLAND) 2023; 23:6778. [PMID: 37571561 PMCID: PMC10422211 DOI: 10.3390/s23156778] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Revised: 07/22/2023] [Accepted: 07/25/2023] [Indexed: 08/13/2023]
Abstract
This work is concerned with the vulnerability of a network industrial control system to cyber-attacks, which is a critical issue nowadays. This is because an attack on a controlled process can damage or destroy it. These attacks use long short-term memory (LSTM) neural networks, which model dynamical processes. This means that the attacker may not know the physical nature of the process; an LSTM network is sufficient to mislead the process operator. Our experimental studies were conducted in an industrial control network containing a magnetic levitation process. The model training, evaluation, and structure selection are described. The chosen LSTM network very well mimicked the considered process. Finally, based on the obtained results, we formulated possible protection methods against the considered types of cyber-attack.
Collapse
|
7
|
Fluchs S, Taştan E, Trumpf T, Horch A, Drath R, Fay A. Traceable Security-by-Design Decisions for Cyber-Physical Systems (CPSs) by Means of Function-Based Diagrams and Security Libraries. SENSORS (BASEL, SWITZERLAND) 2023; 23:5547. [PMID: 37420712 DOI: 10.3390/s23125547] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Revised: 06/05/2023] [Accepted: 06/08/2023] [Indexed: 07/09/2023]
Abstract
"Security by design" is the term for shifting cybersecurity considerations from a system's end users to its engineers. To reduce the end users' workload for addressing security during the systems operation phase, security decisions need to be made during engineering, and in a way that is traceable for third parties. However, engineers of cyber-physical systems (CPSs) or, more specifically, industrial control systems (ICSs) typically neither have the security expertise nor time for security engineering. The security-by-design decisions method presented in this work aims to enable them to identify, make, and substantiate security decisions autonomously. Core features of the method are a set of function-based diagrams as well as libraries of typical functions and their security parameters. The method, implemented as a software demonstrator, is validated in a case study with the specialist for safety-related automation solutions HIMA, and the results show that the method enables engineers to identify and make security decisions they may not have made (consciously) otherwise, and quickly and with little security expertise. The method is also well suited to make security-decision-making knowledge available to less experienced engineers. This means that with the security-by-design decisions method, more people can contribute to a CPS's security by design in less time.
Collapse
|
8
|
Pinto A, Herrera LC, Donoso Y, Gutierrez JA. Survey on Intrusion Detection Systems Based on Machine Learning Techniques for the Protection of Critical Infrastructure. SENSORS (BASEL, SWITZERLAND) 2023; 23:2415. [PMID: 36904618 PMCID: PMC10007329 DOI: 10.3390/s23052415] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/22/2022] [Revised: 02/16/2023] [Accepted: 02/20/2023] [Indexed: 06/18/2023]
Abstract
Industrial control systems (ICSs), supervisory control and data acquisition (SCADA) systems, and distributed control systems (DCSs) are fundamental components of critical infrastructure (CI). CI supports the operation of transportation and health systems, electric and thermal plants, and water treatment facilities, among others. These infrastructures are not insulated anymore, and their connection to fourth industrial revolution technologies has expanded the attack surface. Thus, their protection has become a priority for national security. Cyber-attacks have become more sophisticated and criminals are able to surpass conventional security systems; therefore, attack detection has become a challenging area. Defensive technologies such as intrusion detection systems (IDSs) are a fundamental part of security systems to protect CI. IDSs have incorporated machine learning (ML) techniques that can deal with broader kinds of threats. Nevertheless, the detection of zero-day attacks and having technological resources to implement purposed solutions in the real world are concerns for CI operators. This survey aims to provide a compilation of the state of the art of IDSs that have used ML algorithms to protect CI. It also analyzes the security dataset used to train ML models. Finally, it presents some of the most relevant pieces of research on these topics that have been developed in the last five years.
Collapse
|
9
|
Jadidi Z, Pal S, Hussain M, Nguyen Thanh K. Correlation-Based Anomaly Detection in Industrial Control Systems. SENSORS (BASEL, SWITZERLAND) 2023; 23:1561. [PMID: 36772600 PMCID: PMC9920746 DOI: 10.3390/s23031561] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Revised: 01/19/2023] [Accepted: 01/28/2023] [Indexed: 06/18/2023]
Abstract
Industrial Control Systems (ICSs) were initially designed to be operated in an isolated network. However, recently, ICSs have been increasingly connected to the Internet to expand their capability, such as remote management. This interconnectivity of ICSs exposes them to cyber-attacks. At the same time, cyber-attacks in ICS networks are different compared to traditional Information Technology (IT) networks. Cyber attacks on ICSs usually involve a sequence of actions and a multitude of devices. However, current anomaly detection systems only focus on local analysis, which misses the correlation between devices and the progress of attacks over time. As a consequence, they lack an effective way to detect attacks at an entire network scale and predict possible future actions of an attack, which is of significant interest to security analysts to identify the weaknesses of their network and prevent similar attacks in the future. To address these two key issues, this paper presents a system-wide anomaly detection solution using recurrent neural networks combined with correlation analysis techniques. The proposed solution has a two-layer analysis. The first layer targets attack detection, and the second layer analyses the detected attack to predict the next possible attack actions. The main contribution of this paper is the proof of the concept implementation using two real-world ICS datasets, SWaT and Power System Attack. Moreover, we show that the proposed solution effectively detects anomalies and attacks on the scale of the entire ICS network.
Collapse
|
10
|
Kim B, Alawami MA, Kim E, Oh S, Park J, Kim H. A Comparative Study of Time Series Anomaly Detection Models for Industrial Control Systems. SENSORS (BASEL, SWITZERLAND) 2023; 23:1310. [PMID: 36772349 PMCID: PMC9921147 DOI: 10.3390/s23031310] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 01/13/2023] [Accepted: 01/19/2023] [Indexed: 06/18/2023]
Abstract
Anomaly detection has been known as an effective technique to detect faults or cyber-attacks in industrial control systems (ICS). Therefore, many anomaly detection models have been proposed for ICS. However, most models have been implemented and evaluated under specific circumstances, which leads to confusion about choosing the best model in a real-world situation. In other words, there still needs to be a comprehensive comparison of state-of-the-art anomaly detection models with common experimental configurations. To address this problem, we conduct a comparative study of five representative time series anomaly detection models: InterFusion, RANSynCoder, GDN, LSTM-ED, and USAD. We specifically compare the performance analysis of the models in detection accuracy, training, and testing times with two publicly available datasets: SWaT and HAI. The experimental results show that the best model results are inconsistent with the datasets. For SWaT, InterFusion achieves the highest F1-score of 90.7% while RANSynCoder achieves the highest F1-score of 82.9% for HAI. We also investigate the effects of the training set size on the performance of anomaly detection models. We found that about 40% of the entire training set would be sufficient to build a model producing a similar performance compared to using the entire training set.
Collapse
|
11
|
Ji X, Wei H, Chen Y, Ji XF, Wu G. A Three-Stage Dynamic Assessment Framework for Industrial Control System Security Based on a Method of W-HMM. SENSORS (BASEL, SWITZERLAND) 2022; 22:2593. [PMID: 35408212 PMCID: PMC9002662 DOI: 10.3390/s22072593] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/04/2022] [Revised: 03/23/2022] [Accepted: 03/25/2022] [Indexed: 06/14/2023]
Abstract
Industrial control systems (ICS) are applied in many fields. Due to the development of cloud computing, artificial intelligence, and big data analysis inducing more cyberattacks, ICS always suffers from the risks. If the risks occur during system operations, corporate capital is endangered. It is crucial to assess the security of ICS dynamically. This paper proposes a dynamic assessment framework for industrial control system security (DAF-ICSS) based on machine learning and takes an industrial robot system as an example. The framework conducts security assessment from qualitative and quantitative perspectives, combining three assessment phases: static identification, dynamic monitoring, and security assessment. During the evaluation, we propose a weighted Hidden Markov Model (W-HMM) to dynamically establish the system's security model with the algorithm of Baum-Welch. To verify the effectiveness of DAF-ICSS, we have compared it with two assessment methods to assess industrial robot security. The comparison result shows that the proposed DAF-ICSS can provide a more accurate assessment. The assessment reflects the system's security state in a timely and intuitive manner. In addition, it can be used to analyze the security impact caused by the unknown types of ICS attacks since it infers the security state based on the explicit state of the system.
Collapse
|
12
|
Improved Mitigation of Cyber Threats in IIoT for Smart Cities: A New-Era Approach and Scheme. SENSORS 2021; 21:s21061976. [PMID: 33799668 PMCID: PMC8001220 DOI: 10.3390/s21061976] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/31/2021] [Revised: 02/28/2021] [Accepted: 03/09/2021] [Indexed: 11/17/2022]
Abstract
Cybersecurity in Industrial Internet of Things (IIoT) has become critical as smart cities are becoming increasingly linked to industrial control systems (ICSs) used in critical infrastructure. Consequently, data-driven security systems for analyzing massive amounts of data generated by smart cities have become essential. A representative method for analyzing large-scale data is the game bot detection approach used in massively multiplayer online role-playing games. We reviewed the literature on bot detection methods to extend the anomaly detection approaches used in bot detection schemes to IIoT fields. Finally, we proposed a process wherein the data envelopment analysis (DEA) model was applied to identify features for efficiently detecting anomalous behavior in smart cities. Experimental results using random forest show that our extracted features based on a game bot can achieve an average F1-score of 0.99903 using 10-fold validation. We confirmed the applicability of the analyzed game-industry methodology to other fields and trained a random forest on the high-efficiency features identified by applying a DEA, obtaining an F1-score of 0.997 using the validation set approach. In this study, an anomaly detection method for analyzing massive smart city data based on a game industry methodology was presented and applied to the ICS dataset.
Collapse
|
13
|
A Survey of Blockchain Enabled Cyber-Physical Systems. SENSORS 2020; 20:s20010282. [PMID: 31947860 PMCID: PMC6983181 DOI: 10.3390/s20010282] [Citation(s) in RCA: 41] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/25/2019] [Revised: 12/23/2019] [Accepted: 12/24/2019] [Indexed: 11/29/2022]
Abstract
Cyber-physical systems (CPS) is a setup that controls and monitors the physical world around us. The advancement of these systems needs to incorporate an unequivocal spotlight on making these systems efficient. Blockchains and their inherent combination of consensus algorithms, distributed data storage, and secure protocols can be utilized to build robustness and reliability in these systems. Blockchain is the underlying technology behind bitcoins and it provides a decentralized framework to validate transactions and ensure that they cannot be modified. By distributing the role of information validation across the network peers, blockchain eliminates the risks associated with a centralized architecture. It is the most secure validation mechanism that is efficient and enables the provision of financial services, thereby giving users more freedom and power. This upcoming technology provides internet users with the capability to create value and authenticate digital information. It has the capability to revolutionize a diverse set of business applications, ranging from sharing economy to data management and prediction markets. In this paper, we present a holistic survey of various applications of CPS where blockchain has been utilized. Smart grids, health-care systems, and industrial production processes are some of the many applications that can benefit from the blockchain technology and will be discussed in the paper.
Collapse
|
14
|
Transmission Scheduling Schemes of Industrial Wireless Sensors for Heterogeneous Multiple Control Systems. SENSORS 2018; 18:s18124284. [PMID: 30563135 PMCID: PMC6308482 DOI: 10.3390/s18124284] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/14/2018] [Revised: 11/30/2018] [Accepted: 12/01/2018] [Indexed: 11/30/2022]
Abstract
The transmission scheduling scheme of wireless networks for industrial control systems is a crucial design component since it directly affects the stability of networked control systems. In this paper, we propose a novel transmission scheduling framework to guarantee the stability of heterogeneous multiple control systems over unreliable wireless channels. Based on the explicit control stability conditions, a constrained optimization problem is proposed to maximize the minimum slack of the stability constraint for the heterogeneous control systems. We propose three transmission scheduling schemes, namely centralized stationary random access, distributed random access, and Lyapunov-based scheduling scheme, to solve the constrained optimization problem with a low computation cost. The three proposed transmission scheduling schemes were evaluated on heterogeneous multiple control systems with different link conditions. One interesting finding is that the proposed centralized Lyapunov-based approach provides almost ideal performance in the context of control stability. Furthermore, the distributed random access is still useful for the small number of links since it also reduces the operational overhead without significantly sacrificing the control performance.
Collapse
|
15
|
Busby JS, Green B, Hutchison D. Analysis of Affordance, Time, and Adaptation in the Assessment of Industrial Control System Cybersecurity Risk. RISK ANALYSIS : AN OFFICIAL PUBLICATION OF THE SOCIETY FOR RISK ANALYSIS 2017; 37:1298-1314. [PMID: 28095596 DOI: 10.1111/risa.12681] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/14/2015] [Revised: 05/10/2016] [Accepted: 07/07/2016] [Indexed: 06/06/2023]
Abstract
Industrial control systems increasingly use standard communication protocols and are increasingly connected to public networks-creating substantial cybersecurity risks, especially when used in critical infrastructures such as electricity and water distribution systems. Methods of assessing risk in such systems have recognized for some time the way in which the strategies of potential adversaries and risk managers interact in defining the risk to which such systems are exposed. But it is also important to consider the adaptations of the systems' operators and other legitimate users to risk controls, adaptations that often appear to undermine these controls, or shift the risk from one part of a system to another. Unlike the case with adversarial risk analysis, the adaptations of system users are typically orthogonal to the objective of minimizing or maximizing risk in the system. We argue that this need to analyze potential adaptations to risk controls is true for risk problems more generally, and we develop a framework for incorporating such adaptations into an assessment process. The method is based on the principle of affordances, and we show how this can be incorporated in an iterative procedure based on raising the minimum period of risk materialization above some threshold. We apply the method in a case study of a small European utility provider and discuss the observations arising from this.
Collapse
|