Classification of Sea Ice Types in Sentinel-1 SAR Data Using Convolutional Neural Networks

Artificial intelligence for geoscience: Progress, challenges, and perspectives

This paper explores the evolution of geoscientific inquiry, tracing the progression from traditional physics-based models to modern data-driven approaches facilitated by significant advancements in artificial intelligence (AI) and data collection techniques. Traditional models, which are grounded in physical and numerical frameworks, provide robust explanations by explicitly reconstructing underlying physical processes. However, their limitations in comprehensively capturing Earth's complexities and uncertainties pose challenges in optimization and real-world applicability. In contrast, contemporary data-driven models, particularly those utilizing machine learning (ML) and deep learning (DL), leverage extensive geoscience data to glean insights without requiring exhaustive theoretical knowledge. ML techniques have shown promise in addressing Earth science-related questions. Nevertheless, challenges such as data scarcity, computational demands, data privacy concerns, and the "black-box" nature of AI models hinder their seamless integration into geoscience. The integration of physics-based and data-driven methodologies into hybrid models presents an alternative paradigm. These models, which incorporate domain knowledge to guide AI methodologies, demonstrate enhanced efficiency and performance with reduced training data requirements. This review provides a comprehensive overview of geoscientific research paradigms, emphasizing untapped opportunities at the intersection of advanced AI techniques and geoscience. It examines major methodologies, showcases advances in large-scale models, and discusses the challenges and prospects that will shape the future landscape of AI in geoscience. The paper outlines a dynamic field ripe with possibilities, poised to unlock new understandings of Earth's complexities and further advance geoscience exploration.

AI4SeaIce: selecting loss functions for automated SAR sea ice concentration charting

For maritime navigation in the Arctic, sea ice charts are an essential tool, which still to this day is drawn manually by professional ice analysts. The total Sea Ice Concentration (SIC) is the primary descriptor of the charts and indicates the fraction of ice in an ocean surface area. Naturally, automating the SIC chart creation is desired. However, the optimal representation of the corresponding machine-learning task is ambivalent and discussed in the community. In this study, we explore the representation with either regressional or classification objectives, each with two different (weighted) loss functions: Mean Square Error and Binary Cross-Entropy, and Categorical Cross-Entropy and the Earth Mover's Distance, respectively. While all models achieve good results they differ as the regression-based models obtain the highest numerical similarity to the reference charts, whereas the classification-optimised models generate results more visually pleasing and consistent. Rescaling the loss functions with inverse class weights improves the performance for intermediate classes at the expense of open water and fully-covered sea ice areas.

Sea Ice–Water Classification of RADARSAT-2 Imagery Based on Residual Neural Networks (ResNet) with Regional Pooling

Sea ice mapping plays an integral role in ship navigation and meteorological modeling in the polar regions. Numerous published studies in sea ice classification using synthetic aperture radar (SAR) have reported high classification rates. However, many of these focus on numerical results based on sample points and ignore the quality of the inferred sea ice maps. We have designed and implemented a novel SAR sea ice classification algorithm where the spatial context, obtained by the unsupervised IRGS segmentation algorithm, is integrated with texture features extracted by a residual neural network (ResNet) and, using regional pooling, classifies ice and water. This algorithm is trained and tested on a published dataset and cross-validated using leave-one-out (LOO) strategy, obtaining an overall accuracy of 99.67% and outperforming several existing algorithms. In addition, visual results show that this new method produces sea ice maps with natural ice–water boundaries and fewer ice and water errors.

Water-Body Segmentation for SAR Images: Past, Current, and Future

Synthetic Aperture Radar (SAR), as a microwave sensor that can sense a target all day or night under all-weather conditions, is of great significance for detecting water resources, such as coastlines, lakes and rivers. This paper reviews literature published in the past 30 years in the field of water body extraction in SAR images, and makes some proposals that the community working with SAR image waterbody extraction should consider. Firstly, this review focuses on the main ideas and characteristics of traditional water body extraction on SAR images, mainly focusing on traditional Machine Learning (ML) methods. Secondly, how Deep Learning (DL) methods are applied and optimized in the task of water-body segmentation for SAR images is summarized from the two levels of pixel and image. We also pay more attention to the most popular networks, such as U-Net and its modified models, and novel networks, such as the Cascaded Fully-Convolutional Network (CFCN) and River-Net. In the end, an in-depth discussion is presented, along with conclusions and future trends, on the limitations and challenges of DL for water-body segmentation.

An Improved Sea Ice Classification Algorithm with Gaofen-3 Dual-Polarization SAR Data Based on Deep Convolutional Neural Networks

The distribution of sea ice is one of the major safety hazards for sea navigation. As human activities in polar regions become more frequent, monitoring and forecasting of sea ice are of great significance. In this paper, we use SAR data from the C-band synthetic aperture radar (SAR) Gaofen-3 satellite in the dual-polarization (VV, VH) fine strip II (FSII) mode of operation to study the Arctic sea ice classification in winter. SAR data we use were taken in the western Arctic Ocean from January to February 2020. We classify the sea ice into four categories, namely new ice (NI), thin first-year ice (tI), thick first-year ice (TI), and old ice (OI), by referring to the ice maps provided by the Canadian Ice Service (CIS). Then, we use the deep learning model MobileNetV3 as the backbone network, input samples of different sizes, and combine the backbone network with multiscale feature fusion methods to build a deep learning model called Multiscale MobileNet (MSMN). Dual-polarization SAR data are used to synthesize pseudocolor images and produce samples of sizes 16 × 16 × 3, 32 × 32 × 3, and 64 × 64 × 3 as input. Ultimately, MSMN can reach over 95% classification accuracy on testing SAR sea ice images. The classification results using only VV polarization or VH polarization data are tested, and it is found that using dual-polarization data could improve the classification accuracy by 10.05% and 9.35%, respectively. When other classification models are trained using the training data from this paper for comparison, the accuracy of MSMN is 4.86% and 1.84% higher on average than that of the model built using convolutional neural networks (CNNs) and ResNet18 model, respectively.

Incorporating Aleatoric Uncertainties in Lake Ice Mapping Using RADARSAT–2 SAR Images and CNNs

With the increasing availability of SAR imagery in recent years, more research is being conducted using deep learning (DL) for the classification of ice and open water; however, ice and open water classification using conventional DL methods such as convolutional neural networks (CNNs) is not yet accurate enough to replace manual analysis for operational ice chart mapping. Understanding the uncertainties associated with CNN model predictions can help to quantify errors and, therefore, guide efforts on potential enhancements using more–advanced DL models and/or synergistic approaches. This paper evaluates an approach for estimating the aleatoric uncertainty [a measure used to identify the noise inherent in data] of CNN probabilities to map ice and open water with a custom loss function applied to RADARSAT–2 HH and HV observations. The images were acquired during the 2014 ice season of Lake Erie and Lake Ontario, two of the five Laurentian Great Lakes of North America. Operational image analysis charts from the Canadian Ice Service (CIS), which are based on visual interpretation of SAR imagery, are used to provide training and testing labels for the CNN model and to evaluate the accuracy of the model predictions. Bathymetry, as a variable that has an impact on the ice regime of lakes, was also incorporated during model training in supplementary experiments. Adding aleatoric loss and bathymetry information improved the accuracy of mapping water and ice. Results are evaluated quantitatively (accuracy metrics) and qualitatively (visual comparisons). Ice and open water scores were improved in some sections of the lakes by using aleatoric loss and including bathymetry. In Lake Erie, the ice score was improved by ∼2 on average in the shallow near–shore zone as a result of better mapping of dark ice (low backscatter) in the western basin. As for Lake Ontario, the open water score was improved by ∼6 on average in the deepest profundal off–shore zone.

SI-STSAR-7: A Large SAR Images Dataset with Spatial and Temporal Information for Classification of Winter Sea Ice in Hudson Bay

Remote sensing satellites have been broadly applied to sea ice monitoring. The substantial increase in satellite imagery provides a large amount of data support for deep learning methods in the sea ice classification field. However, there is a lack of public remote sensing datasets to facilitate sea ice classification with spatial and temporal information and to benchmark the deep learning methods. In this paper, we provide a labeled large sea ice dataset derived from time-series sentinel-1 SAR images, dubbed SI-STSAR-7, and a validated dataset construction method for sea ice classification research. The SI-STSAR-7 dataset includes seven different sea ice types corresponding to different sea ice development stages in Hudson Bay during winter, and its samples are time sequences of SAR image patches in order to embody the differences of backscattering intensity and textures between different sea ice types, as well as the change of sea ice with time. We construct the dataset by first performing noise reduction and mitigation of incidence angle dependence on SAR images, and then producing data samples and labeling them based on our proposed sample-producing principles and the weekly regional ice charts provided by Canadian Ice Service. Three baseline classification methods are developed on SI-STSAR-7 to establish benchmarks, which are evaluated with accuracy and kappa coefficient. The sample-producing principles are verified through experiments. Based on the experimental results, sea ice classification can be implemented well on SI-STSAR-7.

Eddies in the Marginal Ice Zone of Fram Strait and Svalbard from Spaceborne SAR Observations in Winter

Here we investigate the intensity of eddy generation and their properties in the marginal ice zone (MIZ) regions of Fram Strait and around Svalbard using spaceborne synthetic aperture radar (SAR) data from Envisat ASAR and Sentinel-1 in winter 2007 and 2018. Analysis of 2039 SAR images allowed identifying 4619 eddy signatures. The number of eddies detected per image per kilometer of MIZ length is similar for both years. Submesoscale and small mesoscale eddies dominate with cyclones detected twice more frequently than anticyclones. Eddy diameters range from 1 to 68 km with mean values of 6 km and 12 km over shallow and deep water, respectively. Mean eddy size grows with increasing ice concentration in the MIZ, yet most eddies are detected at the ice edge and where the ice concentration is below 20%. The fraction of sea ice trapped in cyclones (53%) is slightly higher than that in anticyclones (48%). The amount of sea ice trapped by a single ‘mean’ eddy is about 40 km2, while the average horizontal retreat of the ice edge due to eddy-induced ice melt is about 0.2–0.5 km·d–1 ± 0.02 km·d–1. Relation of eddy occurrence to background currents and winds is also discussed.

A Two-Round Weight Voting Strategy-Based Ensemble Learning Method for Sea Ice Classification of Sentinel-1 Imagery

Sea ice information in the Arctic region is essential for climatic change monitoring and ship navigation. Although many sea ice classification methods have been put forward, the accuracy and usability of classification systems can still be improved. In this paper, a two-round weight voting strategy-based ensemble learning method is proposed for refining sea ice classification. The proposed method includes three main steps. (1) The preferable features of sea ice are constituted by polarization features (HH, HV, HH/HV) and the top six GLCM-derived texture features via a random forest. (2) The initial classification maps can then be generated by an ensemble learning method, which includes six base classifiers (NB, DT, KNN, LR, ANN, and SVM). The tuned voting weights by a genetic algorithm are employed to obtain the category score matrix and, further, the first coarse classification result. (3) Some pixels may be misclassified due to their corresponding numerically close score value. By introducing an experiential score threshold, each pixel is identified as a fuzzy or an explicit pixel. The fuzzy pixels can then be further rectified based on the local similarity of the neighboring explicit pixels, thereby yielding the final precise classification result. The proposed method was examined on 18 Sentinel-1 EW images, which were captured in the Northeast Passage from November 2019 to April 2020. The experiments show that the proposed method can effectively maintain the edge profile of sea ice and restrain noise from SAR. It is superior to the current mainstream ensemble learning algorithms with the overall accuracy reaching 97%. The main contribution of this study is proposing a superior weight voting strategy in the ensemble learning method for sea ice classification of Sentinel-1 imagery, which is of great significance for guiding secure ship navigation and ice hazard forecasting in winter.

Sea Ice Image Classification Based on Heterogeneous Data Fusion and Deep Learning

Sea ice is one of the typical causes of marine disasters. Sea ice image classification is an important component of sea ice detection. Optical data contain rich spectral information, but they do not allow one to easily distinguish between ground objects with a similar spectrum and foreign objects with the same spectrum. Synthetic aperture radar (SAR) data contain rich texture information, but the data usually have a single source. The limitation of single-source data is that they do not allow for further improvements of the accuracy of remote sensing sea ice classification. In this paper, we propose a method for sea ice image classification based on deep learning and heterogeneous data fusion. Utilizing the advantages of convolutional neural networks (CNNs) in terms of depth feature extraction, we designed a deep learning network structure for SAR and optical images and achieve sea ice image classification through feature extraction and a feature-level fusion of heterogeneous data. For the SAR images, the improved spatial pyramid pooling (SPP) network was used and texture information on sea ice at different scales was extracted by depth. For the optical data, multi-level feature information on sea ice such as spatial and spectral information on different types of sea ice was extracted through a path aggregation network (PANet), which enabled low-level features to be fully utilized due to the gradual feature extraction of the convolution neural network. In order to verify the effectiveness of the method, two sets of heterogeneous sentinel satellite data were used for sea ice classification in the Hudson Bay area. The experimental results show that compared with the typical image classification methods and other heterogeneous data fusion methods, the method proposed in this paper fully integrates multi-scale and multi-level texture and spectral information from heterogeneous data and achieves a better classification effect (96.61%, 95.69%).

MYI Floes Identification Based on the Texture and Shape Feature from Dual-Polarized Sentinel-1 Imagery

The Northwest Passage (NWP) in the Arctic is usually covered with hazardous multi-year ice (MYI) and seasonal first-year ice (FYI) in winter, with possible thin ice and open-water areas during transition seasons. Ice classification is important for both marine navigation and climate change studies. Satellite-based Synthetic Aperture Radar (SAR) systems have shown advantages of retrieving this information. Operational ice mapping relies on visual analysis of SAR images along with ancillary data. However, these maps estimate ice types and concentrations within large-size polygons of a few tens or hundreds of kilometers, which are subjectively identified and selected by analysts. This study aims at developing an automated algorithm to identify individual MYI floes from SAR images then classify the rest of the image as FYI and other ice types. The algorithm identifies the MYI floes using extended-maximum operator, morphological image processing, and a few geometrical features. Classifying the rest of the image uses texture and neural network model. The input data is a set of Sentinel-1 A/B Extended Wide (EW) mode images, acquired between September and March 2016–2019. Although the overall accuracy (for all type classification) from the new method scored 93.26%, the accuracy from using the texture classifier only was 75.81%. The kappa coefficient from the former was higher than the latter by 0.25. Compared with the operational ice charts from the Canadian Ice Service, ice type maps from the new method show better distribution of MYI at the fine scale of individual floes. Comparison against MYI concentration from two automated algorithms that use a combination of coarse-resolution passive and active microwave data also confirms the advantage of resolving MYI floes from the fine-resolution SAR.

An Improved Traffic Congestion Monitoring System Based on Federated Learning

This study introduces a software-based traffic congestion monitoring system. The transportation system controls the traffic between cities all over the world. Traffic congestion happens not only in cities, but also on highways and other places. The current transportation system is not satisfactory in the area without monitoring. In order to improve the limitations of the current traffic system in obtaining road data and expand its visual range, the system uses remote sensing data as the data source for judging congestion. Since some remote sensing data needs to be kept confidential, this is a problem to be solved to effectively protect the safety of remote sensing data during the deep learning training process. Compared with the general deep learning training method, this study provides a federated learning method to identify vehicle targets in remote sensing images to solve the problem of data privacy in the training process of remote sensing data. The experiment takes the remote sensing image data sets of Los Angeles Road and Washington Road as samples for training, and the training results can achieve an accuracy of about 85%, and the estimated processing time of each image can be as low as 0.047 s. In the final experimental results, the system can automatically identify the vehicle targets in the remote sensing images to achieve the purpose of detecting congestion.