1
|
Butail S, Bhattacharya A, Porfiri M. Estimating hidden relationships in dynamical systems: Discovering drivers of infection rates of COVID-19. CHAOS (WOODBURY, N.Y.) 2024; 34:033117. [PMID: 38457848 DOI: 10.1063/5.0156338] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/28/2023] [Accepted: 02/12/2024] [Indexed: 03/10/2024]
Abstract
Discovering causal influences among internal variables is a fundamental goal of complex systems research. This paper presents a framework for uncovering hidden relationships from limited time-series data by combining methods from nonlinear estimation and information theory. The approach is based on two sequential steps: first, we reconstruct a more complete state of the underlying dynamical system, and second, we calculate mutual information between pairs of internal state variables to detail causal dependencies. Equipped with time-series data related to the spread of COVID-19 from the past three years, we apply this approach to identify the drivers of falling and rising infections during the three main waves of infection in the Chicago metropolitan region. The unscented Kalman filter nonlinear estimation algorithm is implemented on an established epidemiological model of COVID-19, which we refine to include isolation, masking, loss of immunity, and stochastic transition rates. Through the systematic study of mutual information between infection rate and various stochastic parameters, we find that increased mobility, decreased mask use, and loss of immunity post sickness played a key role in rising infections, while falling infections were controlled by masking and isolation.
Collapse
Affiliation(s)
- S Butail
- Department of Mechanical Engineering, Northern Illinois University, DeKalb, Illinois 60115, USA
| | - A Bhattacharya
- Department of Mechanical Engineering, Northern Illinois University, DeKalb, Illinois 60115, USA
| | - M Porfiri
- Center for Urban Science and Progress, Department of Mechanical and Aerospace Engineering, and Department of Biomedical Engineering, Tandon School of Engineering, New York University, Brooklyn, New York 11201, USA
| |
Collapse
|
2
|
Spatiotemporal Dynamics of NDVI, Soil Moisture and ENSO in Tropical South America. REMOTE SENSING 2022. [DOI: 10.3390/rs14112521] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
We evaluated the coupled dynamics of vegetation dynamics (NDVI) and soil moisture (SMOS) at monthly resolution over different regions of tropical South America and the effects of the Eastern Pacific (EP) and the Central Pacific (CP) El Niño–Southern Oscillation (ENSO) events. We used linear Pearson cross-correlation, wavelet and cross wavelet analysis (CWA) and three nonlinear causality methods: ParrCorr, GPDC and PCMCIplus. Results showed that NDVI peaks when SMOS is transitioning from maximum to minimum monthly values, which confirms the role of SMOS in the hydrological dynamics of the Amazonian greening up during the dry season. Linear correlations showed significant positive values when SMOS leads NDVI by 1–3 months. Wavelet analysis evidenced strong 12- and 64-month frequency bands throughout the entire record length, in particular for SMOS, whereas the CWA analyses indicated that both variables exhibit a strong coherency at a wide range of frequency bands from 2 to 32 months. Linear and nonlinear causality measures also showed that ENSO effects are greater on SMOS. Lagged cross-correlations displayed that western (eastern) regions are more associated with the CP (EP), and that the effects of ENSO manifest as a travelling wave over time, from northwest (earlier) to southeast (later) over tropical South America and the Amazon River basin. The ParrCorr and PCMCIplus methods produced the most coherent results, and allowed us to conclude that: (1) the nonlinear temporal persistence (memory) of soil moisture is stronger than that of NDVI; (2) the existence of two-way nonlinear causalities between NDVI and SMOS; (3) diverse causal links between both variables and the ENSO indices: CP (7/12 with ParrCorr; 6/12 with PCMCIplus), and less with EP (5/12 with ParrCorr; 3/12 with PCMCIplus).
Collapse
|
3
|
Kalemkerian J, Fernández D. An independence test based on recurrence rates. An empirical study and applications to real data. COMMUN STAT-SIMUL C 2022. [DOI: 10.1080/03610918.2022.2037637] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
Affiliation(s)
- Juan Kalemkerian
- Centro de Matemática, Facultad de Ciencias, Universidad de la República, Montevideo, Uruguay
| | - Diego Fernández
- Facultad de Ciencias Económicas y Administración, Universidad de la República, Montevideo, Uruguay
| |
Collapse
|
4
|
Recurrence-Based Synchronization Analysis of Weakly Coupled Bursting Neurons Under External ELF Fields. ENTROPY 2022; 24:e24020235. [PMID: 35205531 PMCID: PMC8871468 DOI: 10.3390/e24020235] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/30/2021] [Revised: 01/27/2022] [Accepted: 02/01/2022] [Indexed: 02/05/2023]
Abstract
We investigate the response characteristics of a two-dimensional neuron model exposed to an externally applied extremely low frequency (ELF) sinusoidal electric field and the synchronization of neurons weakly coupled with gap junction. We find, by numerical simulations, that neurons can exhibit different spiking patterns, which are well observed in the structure of the recurrence plot (RP). We further study the synchronization between weakly coupled neurons in chaotic regimes under the influence of a weak ELF electric field. In general, detecting the phases of chaotic spiky signals is not easy by using standard methods. Recurrence analysis provides a reliable tool for defining phases even for noncoherent regimes or spiky signals. Recurrence-based synchronization analysis reveals that, even in the range of weak coupling, phase synchronization of the coupled neurons occurs and, by adding an ELF electric field, this synchronization increases depending on the amplitude of the externally applied ELF electric field. We further suggest a novel measure for RP-based phase synchronization analysis, which better takes into account the probabilities of recurrences.
Collapse
|
5
|
Braun T, Fernandez CN, Eroglu D, Hartland A, Breitenbach SFM, Marwan N. Sampling rate-corrected analysis of irregularly sampled time series. Phys Rev E 2022; 105:024206. [PMID: 35291153 DOI: 10.1103/physreve.105.024206] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2021] [Accepted: 02/04/2022] [Indexed: 06/14/2023]
Abstract
The analysis of irregularly sampled time series remains a challenging task requiring methods that account for continuous and abrupt changes of sampling resolution without introducing additional biases. The edit distance is an effective metric to quantitatively compare time series segments of unequal length by computing the cost of transforming one segment into the other. We show that transformation costs generally exhibit a nontrivial relationship with local sampling rate. If the sampling resolution undergoes strong variations, this effect impedes unbiased comparison between different time episodes. We study the impact of this effect on recurrence quantification analysis, a framework that is well suited for identifying regime shifts in nonlinear time series. A constrained randomization approach is put forward to correct for the biased recurrence quantification measures. This strategy involves the generation of a type of time series and time axis surrogates which we call sampling-rate-constrained (SRC) surrogates. We demonstrate the effectiveness of the proposed approach with a synthetic example and an irregularly sampled speleothem proxy record from Niue island in the central tropical Pacific. Application of the proposed correction scheme identifies a spurious transition that is solely imposed by an abrupt shift in sampling rate and uncovers periods of reduced seasonal rainfall predictability associated with enhanced El Niño-Southern Oscillation and tropical cyclone activity.
Collapse
Affiliation(s)
- Tobias Braun
- Potsdam Institute for Climate Impact Research (PIK), Member of the Leibniz Association, 14473 Potsdam, Germany
| | - Cinthya N Fernandez
- Institute for Geology, Mineralogy and Geophysics Ruhr-Universität Bochum, 44801 Bochum, Germany
| | - Deniz Eroglu
- Faculty of Engineering and Natural Sciences, Kadir Has University, 34083 Istanbul, Turkey
| | - Adam Hartland
- Environmental Research Institute, School of Science, University of Waikato, Hamilton, Waikato 3240, New Zealand
| | - Sebastian F M Breitenbach
- Department of Geography and Environmental Sciences, Northumbria University, Newcastle upon Tyne NE1 8ST, United Kingdom
| | - Norbert Marwan
- Potsdam Institute for Climate Impact Research (PIK), Member of the Leibniz Association, 14473 Potsdam, Germany
| |
Collapse
|
6
|
Abstract
Nonlinear time series analysis gained prominence from the late 1980s on, primarily because of its ability to characterize, analyze, and predict nontrivial features in data sets that stem from a wide range of fields such as finance, music, human physiology, cognitive science, astrophysics, climate, and engineering. More recently, recurrence plots, initially proposed as a visual tool for the analysis of complex systems, have proven to be a powerful framework to quantify and reveal nontrivial dynamical features in time series data. This tutorial review provides a brief introduction to the fundamentals of nonlinear time series analysis, before discussing in greater detail a few (out of the many existing) approaches of recurrence plot-based analysis of time series. In particular, it focusses on recurrence plot-based measures which characterize dynamical features such as determinism, synchronization, and regime changes. The concept of surrogate-based hypothesis testing, which is crucial to drawing any inference from data analyses, is also discussed. Finally, the presented recurrence plot approaches are applied to two climatic indices related to the equatorial and North Pacific regions, and their dynamical behavior and their interrelations are investigated.
Collapse
|
7
|
Boldini A, Karakaya M, Ruiz Marín M, Porfiri M. Application of symbolic recurrence to experimental data, from firearm prevalence to fish swimming. CHAOS (WOODBURY, N.Y.) 2019; 29:113128. [PMID: 31779365 DOI: 10.1063/1.5119883] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/14/2019] [Accepted: 10/23/2019] [Indexed: 05/28/2023]
Abstract
Recurrence plots and recurrence quantification analysis are powerful tools to study the behavior of dynamical systems. What we learn through these tools is typically determined by the choice of a distance threshold in the phase space, which introduces arbitrariness in the definition of recurrence. Not only does symbolic recurrence overcome this difficulty, but also it offers a richer representation that book-keeps the recurrent portions of the phase space. Using symbolic recurrences, we can construct recurrence plots, perform quantification analysis, and examine causal links between dynamical systems from their time-series. Although previous efforts have demonstrated the feasibility of such a symbolic framework on synthetic data, the study of real time-series remains elusive. Here, we seek to bridge this gap by systematically examining a wide range of experimental datasets, from firearm prevalence and media coverage in the United States to the effect of sex on the interaction of swimming fish. This work offers a compelling demonstration of the potential of symbolic recurrence in the study of real-world applications across different research fields while providing a computer code for researchers to perform their own time-series explorations.
Collapse
Affiliation(s)
- Alain Boldini
- Department of Mechanical and Aerospace Engineering, New York University, Tandon School of Engineering, Brooklyn, New York 11201, USA
| | - Mert Karakaya
- Department of Mechanical and Aerospace Engineering, New York University, Tandon School of Engineering, Brooklyn, New York 11201, USA
| | - Manuel Ruiz Marín
- Department of Quantitative Methods, Law and Modern Languages, Technical University of Cartagena, 30201 Murcia, Spain
| | - Maurizio Porfiri
- Department of Mechanical and Aerospace Engineering, New York University, Tandon School of Engineering, Brooklyn, New York 11201, USA
| |
Collapse
|
8
|
Maksimenko VA, Frolov NS, Hramov AE, Runnova AE, Grubov VV, Kurths J, Pisarchik AN. Neural Interactions in a Spatially-Distributed Cortical Network During Perceptual Decision-Making. Front Behav Neurosci 2019; 13:220. [PMID: 31607873 PMCID: PMC6769171 DOI: 10.3389/fnbeh.2019.00220] [Citation(s) in RCA: 34] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2019] [Accepted: 09/05/2019] [Indexed: 01/11/2023] Open
Abstract
Behavioral experiments evidence that attention is not maintained at a constant level, but fluctuates with time. Recent studies associate such fluctuations with dynamics of attention-related cortical networks, however the exact mechanism remains unclear. To address this issue, we consider functional neuronal interactions during the accomplishment of a reaction time (RT) task which requires sustained attention. The participants are subjected to a binary classification of a large number of presented ambiguous visual stimuli with different degrees of ambiguity. Generally, high ambiguity causes high RT and vice versa. However, we demonstrate that RT fluctuates even when the stimulus ambiguity remains unchanged. The analysis of neuronal activity reveals that the subject's behavioral response is preceded by the formation of a distributed functional network in the β-frequency band. This network is characterized by high connectivity in the frontal cortex and supposed to subserve a decision-making process. We show that neither the network structure nor the duration of its formation depend on RT and stimulus ambiguity. In turn, RT is related to the moment of time when the β-band functional network emerges. We hypothesize that RT is affected by the processes preceding the decision-making stage, e.g., encoding visual sensory information and extracting decision-relevant features from raw sensory information.
Collapse
Affiliation(s)
- Vladimir A Maksimenko
- Neuroscience and Cognitive Technology Laboratory, Center for Technologies in Robotics and Mechatronics Components, Innopolis University, Innopolis, Russia
| | - Nikita S Frolov
- Neuroscience and Cognitive Technology Laboratory, Center for Technologies in Robotics and Mechatronics Components, Innopolis University, Innopolis, Russia
| | - Alexander E Hramov
- Neuroscience and Cognitive Technology Laboratory, Center for Technologies in Robotics and Mechatronics Components, Innopolis University, Innopolis, Russia
| | - Anastasia E Runnova
- Neuroscience and Cognitive Technology Laboratory, Center for Technologies in Robotics and Mechatronics Components, Innopolis University, Innopolis, Russia
| | - Vadim V Grubov
- Neuroscience and Cognitive Technology Laboratory, Center for Technologies in Robotics and Mechatronics Components, Innopolis University, Innopolis, Russia
| | - Jürgen Kurths
- Research Domain IV "Complexity Science", Potsdam Institute for Climate Impact Research, Potsdam, Germany.,Department of Physics, Humboldt University, Berlin, Germany.,Faculty of Biology, Saratov State University, Saratov, Russia
| | - Alexander N Pisarchik
- Neuroscience and Cognitive Technology Laboratory, Center for Technologies in Robotics and Mechatronics Components, Innopolis University, Innopolis, Russia.,Center for Biomedical Technology, Technical University of Madrid, Madrid, Spain
| |
Collapse
|
9
|
Porfiri M, Ruiz Marín M. Transfer entropy on symbolic recurrences. CHAOS (WOODBURY, N.Y.) 2019; 29:063123. [PMID: 31266323 DOI: 10.1063/1.5094900] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/06/2019] [Accepted: 06/03/2019] [Indexed: 05/28/2023]
Abstract
Recurrence quantification analysis offers a powerful framework to investigate complexity in dynamical systems. While several studies have demonstrated the possibility of multivariate recurrence quantification analysis, information-theoretic tools for the discovery of causal links remain elusive. Particularly enticing is to formulate information-theoretic tools on symbolic recurrence plots, which alleviate some of the methodological challenges of traditional recurrence plots and offer a richer representation of recurrences. Toward this aim, we establish a probability space in which we ground a theory of information that encodes information in the recurrences of the symbols. We introduce transfer entropy on symbolic recurrences as a tool to guide the inference of the strength and direction of the interaction between dynamical systems. We demonstrate statistically reliable discovery of causal links on synthetic and experimental time series, from only two time series or a larger dataset with multiple realizations. The proposed approach brings together recurrence plots, information theory, and symbolic dynamics to empower researchers and practitioners with effective means to visualize and quantify interactions in dynamical systems.
Collapse
Affiliation(s)
- Maurizio Porfiri
- Department of Mechanical and Aerospace Engineering and Department of Biomedical Engineering, New York University, Tandon School of Engineering, Brooklyn, New York 11201, USA
| | - Manuel Ruiz Marín
- Department of Quantitative Methods, Law and Modern Languages, Technical University of Cartagena, Murcia, Spain
| |
Collapse
|
10
|
Maksimenko VA, Hramov AE, Frolov NS, Lüttjohann A, Nedaivozov VO, Grubov VV, Runnova AE, Makarov VV, Kurths J, Pisarchik AN. Increasing Human Performance by Sharing Cognitive Load Using Brain-to-Brain Interface. Front Neurosci 2018; 12:949. [PMID: 30631262 PMCID: PMC6315120 DOI: 10.3389/fnins.2018.00949] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2018] [Accepted: 11/29/2018] [Indexed: 11/13/2022] Open
Abstract
Brain-computer interfaces (BCIs) attract a lot of attention because of their ability to improve the brain's efficiency in performing complex tasks using a computer. Furthermore, BCIs can increase human's performance not only due to human-machine interactions, but also thanks to an optimal distribution of cognitive load among all members of a group working on a common task, i.e., due to human-human interaction. The latter is of particular importance when sustained attention and alertness are required. In every day practice, this is a common occurrence, for example, among office workers, pilots of a military or a civil aircraft, power plant operators, etc. Their routinely work includes continuous monitoring of instrument readings and implies a heavy cognitive load due to processing large amounts of visual information. In this paper, we propose a brain-to-brain interface (BBI) which estimates brain states of every participant and distributes a cognitive load among all members of the group accomplishing together a common task. The BBI allows sharing the whole workload between all participants depending on their current cognitive performance estimated from their electrical brain activity. We show that the team efficiency can be increased due to redistribution of the work between participants so that the most difficult workload falls on the operator who exhibits maximum performance. Finally, we demonstrate that the human-to-human interaction is more efficient in the presence of a certain delay determined by brain rhythms. The obtained results are promising for the development of a new generation of communication systems based on neurophysiological brain activity of interacting people. Such BBIs will distribute a common task between all group members according to their individual physical conditions.
Collapse
Affiliation(s)
- Vladimir A Maksimenko
- REC "Artificial Intelligence Systems and Neurotechnology", Yuri Gagarin State Technical University of Saratov, Saratov, Russia
| | - Alexander E Hramov
- REC "Artificial Intelligence Systems and Neurotechnology", Yuri Gagarin State Technical University of Saratov, Saratov, Russia
| | - Nikita S Frolov
- REC "Artificial Intelligence Systems and Neurotechnology", Yuri Gagarin State Technical University of Saratov, Saratov, Russia
| | | | - Vladimir O Nedaivozov
- REC "Artificial Intelligence Systems and Neurotechnology", Yuri Gagarin State Technical University of Saratov, Saratov, Russia
| | - Vadim V Grubov
- REC "Artificial Intelligence Systems and Neurotechnology", Yuri Gagarin State Technical University of Saratov, Saratov, Russia
| | - Anastasia E Runnova
- REC "Artificial Intelligence Systems and Neurotechnology", Yuri Gagarin State Technical University of Saratov, Saratov, Russia
| | - Vladimir V Makarov
- REC "Artificial Intelligence Systems and Neurotechnology", Yuri Gagarin State Technical University of Saratov, Saratov, Russia
| | - Jürgen Kurths
- Potsdam Institute for Climate Impact Research, Potsdam, Germany.,Department of Physics, Humboldt University, Berlin, Germany.,Institute for Complex Systems and Mathematical Biology, University of Aberdeen, Aberdeen, United Kingdom
| | - Alexander N Pisarchik
- REC "Artificial Intelligence Systems and Neurotechnology", Yuri Gagarin State Technical University of Saratov, Saratov, Russia.,Center for Biomedical Technology, Technical University of Madrid, Madrid, Spain
| |
Collapse
|
11
|
Builes-Jaramillo A, Ramos AMT, Poveda G. Atmosphere-Land Bridge between the Pacific and Tropical North Atlantic SST's through the Amazon River basin during the 2005 and 2010 droughts. CHAOS (WOODBURY, N.Y.) 2018; 28:085705. [PMID: 30180604 DOI: 10.1063/1.5020502] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/22/2017] [Accepted: 02/15/2018] [Indexed: 06/08/2023]
Abstract
The present work uses a new approach to causal inference between complex systems called the Recurrence Measure of Conditional Dependence (RMCD) based on the recurrence plots theory, in order to study the role of the Amazon River basin (AM) as a land-atmosphere bridge between the Niño 3.0 region in the Pacific Ocean and the Tropical North Atlantic. Two anomalous droughts in the Amazon River basin were selected, one mainly attributed to the warming of the Tropical North Atlantic (2005) and the other to a warm phase of El Niño-Southern Oscillation (2010). The results of the RMCD analysis evidence the distinctive behavior in the causal information transferred between the two oceanic regions during the two extreme droughts, suggesting that the land-atmosphere bridge operating over the AM is an active hydroclimate mechanism at interannual timescales, and that the RMCD analysis may be an ancillary resort to complement early warning systems.
Collapse
Affiliation(s)
- Alejandro Builes-Jaramillo
- Department of Geosciences and Environment, Facultad de Minas, Universidad Nacional de Colombia, Sede Medellín, Carrera 80 x Calle 65, 050041, Medellín, Colombia
| | - Antônio M T Ramos
- National Institute for Space Research-INPE, 12227-010 São José dos Campos, São Paulo, Brazil
| | - Germán Poveda
- Department of Geosciences and Environment, Facultad de Minas, Universidad Nacional de Colombia, Sede Medellín, Carrera 80 x Calle 65, 050041, Medellín, Colombia
| |
Collapse
|
12
|
Amigó JM, Hirata Y. Detecting directional couplings from multivariate flows by the joint distance distribution. CHAOS (WOODBURY, N.Y.) 2018; 28:075302. [PMID: 30070509 DOI: 10.1063/1.5010779] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
The identification of directional couplings (or drive-response relationships) in the analysis of interacting nonlinear systems is an important piece of information to understand their dynamics. This task is especially challenging when the analyst's knowledge of the systems reduces virtually to time series of observations. Spurred by the success of Granger causality in econometrics, the study of cause-effect relationships (not to be confounded with statistical correlations) was extended to other fields, thus favoring the introduction of further tools such as transfer entropy. Currently, the research on old and new causality tools along with their pitfalls and applications in ever more general situations is going through a time of much activity. In this paper, we re-examine the method of the joint distance distribution to detect directional couplings between two multivariate flows. This method is based on the forced Takens theorem, and, more specifically, it exploits the existence of a continuous mapping from the reconstructed attractor of the response system to the reconstructed attractor of the driving system, an approach that is increasingly drawing the attention of the data analysts. The numerical results with Lorenz and Rössler oscillators in three different interaction networks (including hidden common drivers) are quite satisfactory, except when phase synchronization sets in. They also show that the method of the joint distance distribution outperforms the lowest dimensional transfer entropy in the cases considered. The robustness of the results to the sampling interval, time series length, observational noise, and metric is analyzed too.
Collapse
Affiliation(s)
- José M Amigó
- Centro de Investigación Operativa, Universidad Miguel Hernández, Avda. de la Universidad s/n, 03202 Elche, Spain
| | - Yoshito Hirata
- Mathematics and Informatics Center, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan and The Institute of Industrial Science, The University of Tokyo, 4-6-1 Komaba, Meguro-ku, Tokyo 153-8505, Japan
| |
Collapse
|