SpectralGPT: Spectral Remote Sensing Foundation Model

The foundation model has recently garnered significant attention due to its potential to revolutionize the field of visual representation learning in a self-supervised manner. While most foundation models are tailored to effectively process RGB images for various visual tasks, there is a noticeable gap in research focused on spectral data, which offers valuable information for scene understanding, especially in remote sensing (RS) applications. To fill this gap, we created for the first time a universal RS foundation model, named SpectralGPT, which is purpose-built to handle spectral RS images using a novel 3D generative pretrained transformer (GPT). Compared to existing foundation models, SpectralGPT 1) accommodates input images with varying sizes, resolutions, time series, and regions in a progressive training fashion, enabling full utilization of extensive RS Big Data; 2) leverages 3D token generation for spatial-spectral coupling; 3) captures spectrally sequential patterns via multi-target reconstruction; 4) trains on one million spectral RS images, yielding models with over 600 million parameters. Our evaluation highlights significant performance improvements with pretrained SpectralGPT models, signifying substantial potential in advancing spectral RS Big Data applications within the field of geoscience across four downstream tasks: single/multi-label scene classification, semantic segmentation, and change detection.

DRBM-ClustNet: A Deep Restricted Boltzmann-Kohonen Architecture for Data Clustering

A Bayesian deep restricted Boltzmann-Kohonen architecture for data clustering termed deep restricted Boltzmann machine (DRBM)-ClustNet is proposed. This core-clustering engine consists of a DRBM for processing unlabeled data by creating new features that are uncorrelated and have large variance with each other. Next, the number of clusters is predicted using the Bayesian information criterion (BIC), followed by a Kohonen network (KN)-based clustering layer. The processing of unlabeled data is done in three stages for efficient clustering of the nonlinearly separable datasets. In the first stage, DRBM performs nonlinear feature extraction by capturing the highly complex data representation by projecting the feature vectors of d dimensions into n dimensions. Most clustering algorithms require the number of clusters to be decided a priori; hence, here, to automate the number of clusters in the second stage, we use BIC. In the third stage, the number of clusters derived from BIC forms the input for the KN, which performs clustering of the feature-extracted data obtained from the DRBM. This method overcomes the general disadvantages of clustering algorithms, such as the prior specification of the number of clusters, convergence to local optima, and poor clustering accuracy on nonlinear datasets. In this research, we use two synthetic datasets, 15 benchmark datasets from the UCI Machine Learning repository, and four image datasets to analyze the DRBM-ClustNet. The proposed framework is evaluated based on clustering accuracy and ranked against other state-of-the-art clustering methods. The obtained results demonstrate that the DRBM-ClustNet outperforms state-of-the-art clustering algorithms.

Comprehensive mapping of lunar surface chemistry by adding Chang'e-5 samples with deep learning

Lunar surface chemistry is essential for revealing petrological characteristics to understand the evolution of the Moon. Existing chemistry mapping from Apollo and Luna returned samples could only calibrate chemical features before 3.0 Gyr, missing the critical late period of the Moon. Here we present major oxides chemistry maps by adding distinctive 2.0 Gyr Chang'e-5 lunar soil samples in combination with a deep learning-based inversion model. The inferred chemical contents are more precise than the Lunar Prospector Gamma-Ray Spectrometer (GRS) maps and are closest to returned samples abundances compared to existing literature. The verification of in situ measurement data acquired by Chang'e 3 and Chang'e 4 lunar rover demonstrated that Chang'e-5 samples are indispensable ground truth in mapping lunar surface chemistry. From these maps, young mare basalt units are determined which can be potential sites in future sample return mission to constrain the late lunar magmatic and thermal history.

Iterative Training Sample Augmentation for Enhancing Land Cover Change Detection Performance With Deep Learning Neural Network

Labeled samples are important in achieving land cover change detection (LCCD) tasks via deep learning techniques with remote sensing images. However, labeling samples for change detection with bitemporal remote sensing images is labor-intensive and time-consuming. Moreover, manually labeling samples between bitemporal images requires professional knowledge for practitioners. To address this problem in this article, an iterative training sample augmentation (ITSA) strategy to couple with a deep learning neural network for improving LCCD performance is proposed here. In the proposed ITSA, we start by measuring the similarity between an initial sample and its four-quarter-overlapped neighboring blocks. If the similarity satisfies a predefined constraint, then a neighboring block will be selected as the potential sample. Next, a neural network is trained with renewed samples and used to predict an intermediate result. Finally, these operations are fused into an iterative algorithm to achieve the training and prediction of a neural network. The performance of the proposed ITSA strategy is verified with some widely used change detection deep learning networks using seven pairs of real remote sensing images. The excellent visual performance and quantitative comparisons from the experiments clearly indicate that detection accuracies of LCCD can be effectively improved when a deep learning network is coupled with the proposed ITSA. For example, compared with some state-of-the-art methods, the quantitative improvement is 0.38%-7.53% in terms of overall accuracy. Moreover, the improvement is robust, generic to both homogeneous and heterogeneous images, and universally adaptive to various neural networks of LCCD. The code will be available at https://github.com/ImgSciGroup/ITSA.

Scheduling-Guided Automatic Processing of Massive Hyperspectral Image Classification on Cloud Computing Architectures

The large data volume and high algorithm complexity of hyperspectral image (HSI) problems have posed big challenges for efficient classification of massive HSI data repositories. Recently, cloud computing architectures have become more relevant to address the big computational challenges introduced in the HSI field. This article proposes an acceleration method for HSI classification that relies on scheduling metaheuristics to automatically and optimally distribute the workload of HSI applications across multiple computing resources on a cloud platform. By analyzing the procedure of a representative classification method, we first develop its distributed and parallel implementation based on the MapReduce mechanism on Apache Spark. The subtasks of the processing flow that can be processed in a distributed way are identified as divisible tasks. The optimal execution of this application on Spark is further formulated as a divisible scheduling framework that takes into account both task execution precedences and task divisibility when allocating the divisible and indivisible subtasks onto computing nodes. The formulated scheduling framework is an optimization procedure that searches for optimized task assignments and partition counts for divisible tasks. Two metaheuristic algorithms are developed to solve this divisible scheduling problem. The scheduling results provide an optimized solution to the automatic processing of HSI big data on clouds, improving the computational efficiency of HSI classification by exploring the parallelism during the parallel processing flow. Experimental results demonstrate that our scheduling-guided approach achieves remarkable speedups by facilitating the automatic processing of HSI classification on Spark, and is scalable to the increasing HSI data volume.

A novel optimal multi-pattern matching method with wildcards for DNA sequence

BACKGROUND:

DNA sequence alignment is one of the most fundamental and important operation to identify which gene family may contain this sequence, pattern matching for DNA sequence has been a fundamental issue in biomedical engineering, biotechnology and health informatics.

OBJECTIVE:

To solve this problem, this study proposes an optimal multi pattern matching with wildcards for DNA sequence.

METHODS:

This proposed method packs the patterns and a sliding window of texts, and the window slides along the given packed text, matching against stored packed patterns.

RESULTS:

Three data sets are used to test the performance of the proposed algorithm, and the algorithm was seen to be more efficient than the competitors because its operation is close to machine language.

CONCLUSIONS:

Theoretical analysis and experimental results both demonstrate that the proposed method outperforms the state-of-the-art methods and is especially effective for the DNA sequence.

Automatic Attribute Profiles

Morphological attribute profiles are multilevel decompositions of images obtained with a sequence of transformations performed by connected operators. They have been extensively employed in performing multi-scale and region-based analysis in a large number of applications. One main, still unresolved, issue is the selection of filter parameters able to provide representative and non-redundant threshold decomposition of the image. This paper presents a framework for the automatic selection of filter thresholds based on Granulometric Characteristic Functions (GCFs). GCFs describe the way that non-linear morphological filters simplify a scene according to a given measure. Since attribute filters rely on a hierarchical representation of an image (e.g., the Tree of Shapes) for their implementation, GCFs can be efficiently computed by taking advantage of the tree representation. Eventually, the study of the GCFs allows the identification of a meaningful set of thresholds. Therefore, a trial and error approach is not necessary for the threshold selection, automating the process and in turn decreasing the computational time. It is shown that the redundant information is reduced within the resulting profiles (a problem of high occurrence, as regards manual selection). The proposed approach is tested on two real remote sensing data sets, and the classification results are compared with strategies present in the literature.

Automatic registration of retina images based on genetic techniques

The aim of this paper is to develop an automatic method for the registration of multitemporal digital images of the fundus of the human retina. The images are acquired from the same patient at different times by a color fundus camera. The proposed approach is based on the application of global optimization techniques to previously extracted maps of curvilinear structures in the images to be registered (such structures being represented by the vessels in the human retina): in particular, a genetic algorithm is used, in order to estimate the optimum transformation between the input and the base image. The algorithm is tested on two different types of data, gray scale and color images, and for both types, images with small changes and with large changes are used. The comparison between the registered images using the implemented method and a manual one points out that the proposed algorithm provides an accurate registration. The convergence to a solution is not possible only when dealing with images taken from very different view-points.

Automatic Retinal Oximetry

PURPOSE

To measure hemoglobin oxygen saturation (SO(2)) in retinal vessels and to test the reproducibility and sensitivity of an automatic spectrophotometric oximeter.

METHODS

Specialized software automatically identifies the retinal blood vessels on fundus images, which are obtained with four different wavelengths of light. The software calculates optical density ratios (ODRs) for each vessel. The reproducibility was evaluated by analyzing five repeated measurements of the same vessels. A linear relationship between SO(2) and ODR was assumed and a linear model derived. After calibration, reproducibility and sensitivity were calculated in terms of SO(2). Systemic hyperoxia (n = 16) was induced in healthy volunteers by changing the O(2) concentration in inhaled air from 21% to 100%.

RESULTS

The automatic software enhanced reproducibility, and the mean SD for repeated measurements was 3.7% for arterioles and 5.3% venules, in terms of percentage of SO(2) (five repeats, 10 individuals). The model derived for calibration was SO(2) = 125 - 142 . ODR. The arterial SO(2) measured 96% +/- 9% (mean +/- SD) during normoxia and 101% +/- 8% during hyperoxia (n = 16). The difference between normoxia and hyperoxia was significant (P = 0.0027, paired t-test). Corresponding numbers for venules were 55% +/- 14% and 78% +/- 15% (P < 0.0001). SO(2) is displayed as a pseudocolor map drawn on fundus images.

CONCLUSIONS

The retinal oximeter is reliable, easy to use, and sensitive to changes in SO(2) when concentration of O(2) in inhaled air is changed.

Parallel principal component neural networks for classification of event-related potential waveforms

Artificial neural networks (ANNs) are discussed in terms of classification of brain auditory event-related potentials (ERPs). A new ANN architecture for the classification of ERPs is proposed. The new architecture is called the parallel principal component neural network (PPCNN). The use of the PPCNN for classification of ERP data obtained from both normal control subjects and chronic schizophrenic patients is discussed. Experimental results are given.