Retrieval of Similar Evolution Patterns from Satellite Image Time Series

Multimodal Satellite Image Time Series Analysis Using GAN-Based Domain Translation and Matrix Profile

The technological development of the remote sensing domain led to the acquisition of satellite image time series (SITS) for Earth Observation (EO) by a variety of sensors. The variability in terms of the characteristics of the satellite sensors requires the existence of algorithms that allow the integration of multiple modalities and the identification of anomalous spatio-temporal evolutions caused by natural hazards. The unsupervised analysis of multimodal SITS proposed in this paper follows a two-step methodology: (i) inter-modality translation and (ii) the identification of anomalies in a change-detection framework. Inter-modality translation is achieved by means of a Generative Adversarial Network (GAN) architecture, whereas, for the identification of anomalies caused by natural hazards, we adapt the task to a similarity search in SITS. In this regard, we provide an extension of the matrix profile concept, which represents an answer to identifying differences and to discovering novelties in time series. Furthermore, the proposed inter-modality translation allows the usage of standard unsupervised clustering approaches (e.g., K-means using the Dynamic Time Warping measure) for mono-modal SITS analysis. The effectiveness of the proposed methodology is shown in two use-case scenarios, namely flooding and landslide events, for which a joint acquisition of Sentinel-1 and Sentinel-2 images is performed.

ERS-1/2 and Sentinel-1 SAR Data Mining for Flood Hazard and Risk Assessment in Lima, Peru

The coastline environment and urban areas of Peru overlooking the Pacific Ocean are among the most affected by El Niño-Southern Oscillation (ENSO) events, and its cascading hazards such as floods, landslides and avalanches. In this work, the complete archives of the European Space Agency (ESA)’s European Remote-Sensing (ERS-1/2) missions and European Commission’s Copernicus Sentinel-1 constellation were screened to select synthetic aperture radar (SAR) images covering the most severe and recent ENSO-related flooding events that affected Lima, the capital and largest city of Peru, in 1997–1998 and 2017–2018. Based on SAR backscatter color composites and ratio maps retrieved from a series of pre-, cross- and post-event SAR pairs, flooded areas were delineated within the Rímac River watershed. These are mostly concentrated along the riverbanks and plain, where low-lying topography and gentle slopes (≤5°), together with the presence of alluvial deposits, also indicate greater susceptibility to flooding. A total of 409 areas (58.50 km2) revealing change were mapped, including 197 changes (32.10 km2) due to flooding-related backscatter variations (flooded areas, increased water flow in the riverbed, and riverbank collapses and damage), and 212 (26.40 km2) due to other processes (e.g., new urban developments, construction of river embankments, other engineering works, vegetation changes). Urban and landscape changes potentially contributing, either detrimentally or beneficially, to flooding susceptibility were identified and considered in the overall assessment of risk. The extent of built-up areas within the basin was mapped by combining information from the 2011 Global Urban Footprint (GUF) produced by the German Aerospace Center (DLR), the Open Street Map (OSM) accessed from the Quantum GIS (QGIS) service, and 2011–2019 very high-resolution optical imagery from Google Earth. The resulting flooding risk map highlights the sectors of potential concern along the Rímac River, should flooding events of equal severity as those captured by SAR images occur in the future.

Non-Linear Template-Based Approach for the Study of Locomotion

The automatic detection of gait events (i.e., Initial Contact (IC) and Final Contact (FC)) is crucial for the characterisation of gait from Inertial Measurements Units. In this article, we present a method for detecting steps (i.e., IC and FC) from signals of gait sequences of individuals recorded with a gyrometer. The proposed approach combines the use of a dictionary of templates and a Dynamic Time Warping (DTW) measure of fit to retrieve these templates into input signals. Several strategies for choosing and learning the adequate templates from annotated data are also described. The method is tested on thirteen healthy subjects and compared to gold standard. Depending of the template choice, the proposed algorithm achieves average errors from 0.01 to 0.03 s for the detection of IC, FC and step duration. Results demonstrate that the use of DTW allows achieving these performances with only one single template. DTW is a convenient tool to perform pattern recognition on gait gyrometer signals. This study paves the way for new step detection methods: it shows that using one single template associated with non-linear deformations may be sufficient to model the gait of healthy subjects.

Efficient Kernel-Based Subsequence Search for Enabling Health Monitoring Services in IoT-Based Home Setting

This paper presents an efficient approach for subsequence search in data streams. The problem consists of identifying coherent repetitions of a given reference time-series, also in the multivariate case, within a longer data stream. The most widely adopted metric to address this problem is Dynamic Time Warping (DTW), but its computational complexity is a well-known issue. In this paper, we present an approach aimed at learning a kernel approximating DTW for efficiently analyzing streaming data collected from wearable sensors, while reducing the burden of DTW computation. Contrary to kernel, DTW allows for comparing two time-series with different length. To enable the use of kernel for comparing two time-series with different length, a feature embedding is required in order to obtain a fixed length vector representation. Each vector component is the DTW between the given time-series and a set of “basis” series, randomly chosen. The approach has been validated on two benchmark datasets and on a real-life application for supporting self-rehabilitation in elderly subjects has been addressed. A comparison with traditional DTW implementations and other state-of-the-art algorithms is provided: results show a slight decrease in accuracy, which is counterbalanced by a significant reduction in computational costs.

Hardware-Accelerated, Short-Term Processing Voice and Nonvoice Sound Recognitions for Electric Equipment Control

We proposed and implemented a sound recognition system for electric equipment control. In recent years, industry 4.0 has propelled a rapid growth in intelligent human–machine interactions. User acoustic voice commands for machine control have been examined the most by researchers. The targeted machine can be controlled through voice without the use of any hand-held device. However, compared with human voice recognition, limited research has been conducted on nonhuman voice (e.g., mewing sounds) or nonvoice sound recognition (e.g., clapping). Processing of such short-term, biometric nonvoice sounds for electric equipment control requires a rapid response with correct recognition. In practice, this could lead to a trade-off between recognition accuracy and processing performance for conventional software-based implementations. Therefore, we realized a field-programmable gate array-based embedded system, such a hardware-accelerated platform, can enhance information processing performance using a dynamic time warping accelerator. Furthermore, information processing was refined for two specific applications (i.e., mewing sounds and clapping) to enhance system performance including recognition accuracy and execution speed. Performance analyses and demonstrations on real products were conducted to validate the proposed system.

An Approach for a Spatial Data Attribute Similarity Measure Based on Granular Computing Closeness

This paper proposes a spatial data attribute similarity measure method based on granular computing closeness. This method uses the distance and membership degree of different index levels of spatial entities to measure the similarity of attributes. It not only reflects the degree of similarity of spatial entity types at different index levels but also reflects the integration similarity between spatial entity types under a comprehensive index. This method embodies the layered idea of granular computing and can provide a basis for spatial problem decision making and for spatial entity classification. Finally, the feasibility and applicability of the method are verified by taking the similarity measure of the land-use type attribute in Guangxi as an example.