An Emulator Toolbox to Approximate Radiative Transfer Models with Statistical Learning

Hyperspectral remote sensing for tobacco quality estimation, yield prediction, and stress detection: A review of applications and methods

Tobacco is an important economic crop and the main raw material of cigarette products. Nowadays, with the increasing consumer demand for high-quality cigarettes, the requirements for their main raw materials are also varying. In general, tobacco quality is primarily determined by the exterior quality, inherent quality, chemical compositions, and physical properties. All these aspects are formed during the growing season and are vulnerable to many environmental factors, such as climate, geography, irrigation, fertilization, diseases and pests, etc. Therefore, there is a great demand for tobacco growth monitoring and near real-time quality evaluation. Herein, hyperspectral remote sensing (HRS) is increasingly being considered as a cost-effective alternative to traditional destructive field sampling methods and laboratory trials to determine various agronomic parameters of tobacco with the assistance of diverse hyperspectral vegetation indices and machine learning algorithms. In light of this, we conduct a comprehensive review of the HRS applications in tobacco production management. In this review, we briefly sketch the principles of HRS and commonly used data acquisition system platforms. We detail the specific applications and methodologies for tobacco quality estimation, yield prediction, and stress detection. Finally, we discuss the major challenges and future opportunities for potential application prospects. We hope that this review could provide interested researchers, practitioners, or readers with a basic understanding of current HRS applications in tobacco production management, and give some guidelines for practical works.

Neural Network Emulation of Synthetic Hyperspectral Sentinel-2-like Imagery with Uncertainty

Hyperspectral satellite imagery provides highly-resolved spectral information for large areas and can provide vital information. However, only a few imaging spectrometer missions are currently in operation. Aiming to generate synthetic satellite-based hyperspectral imagery potentially covering any region, we explored the possibility of applying statistical learning, i.e. emulation. Based on the relationship of a Sentinel-2 (S2) scene and a hyperspectral HyPlant airborne image, this work demonstrates the possibility to emulate a hyperspectral S2-like image. We tested the role of different machine learning regression algorithms (MLRA) and varied the image-extracted training dataset size. We found superior performance of Neural Network (NN) as opposed to the other algorithms when trained with large datasets (up to 100'000 samples). The developed emulator was then applied to the L2A (bottom-of-atmosphere reflectance) S2 subset, and the obtained S2-like hyperspectral reflectance scene was evaluated. The validation of emulated against reference spectra demonstrated the potential of the technique. R 2 values between 0.75-0.9 and NRMSE between 2-5% across the full 402-2356 nm range were obtained. Moreover, epistemic uncertainty is obtained using the dropout technique, revealing spatial fidelity of the emulated scene. We obtained highest SD values of 0.05 (CV of 8%) in clouds and values below 0.01 (CV of 7%) in vegetation land covers. Finally, the emulator was applied to an entire S2 tile (5490x5490 pixels) to generate a hyperspectral reflectance datacube with the texture of S2 (60Gb, at a speed of 0.14sec/10000pixels). As the emulator can convert any S2 tile into a hyperspectral image, such scenes give perspectives how future satellite imaging spectroscopy will look like.

Introducing ARTMO's Machine-Learning Classification Algorithms Toolbox: Application to Plant-Type Detection in a Semi-Steppe Iranian Landscape

Accurate plant-type (PT) detection forms an important basis for sustainable land management maintaining biodiversity and ecosystem services. In this sense, Sentinel-2 satellite images of the Copernicus program offer spatial, spectral, temporal, and radiometric characteristics with great potential for mapping and monitoring PTs. In addition, the selection of a best-performing algorithm needs to be considered for obtaining PT classification as accurate as possible. To date, no freely downloadable toolbox exists that brings the diversity of the latest supervised machine-learning classification algorithms (MLCAs) together into a single intuitive user-friendly graphical user interface (GUI). To fill this gap and to facilitate and automate the usage of MLCAs, here we present a novel GUI software package that allows systematically training, validating, and applying pixel-based MLCA models to remote sensing imagery. The so-called MLCA toolbox has been integrated within ARTMO's software framework developed in Matlab which implements most of the state-of-the-art methods in the machine learning community. To demonstrate its utility, we chose a heterogeneous case study scene, a landscape in Southwest Iran to map PTs. In this area, four main PTs were identified, consisting of shrub land, grass land, semi-shrub land, and shrub land-grass land vegetation. Having developed 21 MLCAs using the same training and validation, datasets led to varying accuracy results. Gaussian process classifier (GPC) was validated as the top-performing classifier, with an overall accuracy (OA) of 90%. GPC follows a Laplace approximation to the Gaussian likelihood under the supervised classification framework, emerging as a very competitive alternative to common MLCAs. Random forests resulted in the second-best performance with an OA of 86%. Two other types of ensemble-learning algorithms, i.e., tree-ensemble learning (bagging) and decision tree (with error-correcting output codes), yielded an OA of 83% and 82%, respectively. Following, thirteen classifiers reported OA between 70% and 80%, and the remaining four classifiers reported an OA below 70%. We conclude that GPC substantially outperformed all classifiers, and thus, provides enormous potential for the classification of a diversity of land-cover types. In addition, its probabilistic formulation provides valuable band ranking information, as well as associated predictive variance at a pixel level. Nevertheless, as these are supervised (data-driven) classifiers, performances depend on the entered training data, meaning that an assessment of all MLCAs is crucial for any application. Our analysis demonstrated the efficacy of ARTMO's MLCA toolbox for an automated evaluation of the classifiers and subsequent thematic mapping.

Emulation of Sun-Induced Fluorescence from Radiance Data Recorded by the HyPlant Airborne Imaging Spectrometer

The retrieval of sun-induced fluorescence (SIF) from hyperspectral radiance data grew to maturity with research activities around the FLuorescence EXplorer satellite mission FLEX, yet full-spectrum estimation methods such as the spectral fitting method (SFM) are computationally expensive. To bypass this computational load, this work aims to approximate the SFM-based SIF retrieval by means of statistical learning, i.e., emulation. While emulators emerged as fast surrogate models of simulators, the accuracy-speedup trade-offs are still to be analyzed when the emulation concept is applied to experimental data. We evaluated the possibility of approximating the SFM-like SIF output directly based on radiance data while minimizing the loss in precision as opposed to SFM-based SIF. To do so, we implemented a double principal component analysis (PCA) dimensionality reduction, i.e., in both input and output, to achieve emulation of multispectral SIF output based on hyperspectral radiance data. We then evaluated systematically: (1) multiple machine learning regression algorithms, (2) number of principal components, (3) number of training samples, and (4) quality of training samples. The best performing SIF emulator was then applied to a HyPlant flight line containing at sensor radiance information, and the results were compared to the SFM SIF map of the same flight line. The emulated SIF map was quasi-instantaneously generated, and a good agreement against the reference SFM map was obtained with a R 2 of 0.88 and NRMSE of 3.77%. The SIF emulator was subsequently applied to 7 HyPlant flight lines to evaluate its robustness and portability, leading to a R 2 between 0.68 and 0.95, and a NRMSE between 6.42% and 4.13%. Emulated SIF maps proved to be consistent while processing time was in the order of 3 min. In comparison, the original SFM needed approximately 78 min to complete the SIF processing. Our results suggest that emulation can be used to efficiently reduce computational loads of SIF retrieval methods.

Crop Nitrogen Retrieval Methods for Simulated Sentinel-2 Data Using In-Field Spectrometer Data

Nitrogen (N) is one of the key nutrients supplied in agricultural production worldwide. Over-fertilization can have negative influences on the field and the regional level (e.g., agro-ecosystems). Remote sensing of the plant N of field crops presents a valuable tool for the monitoring of N flows in agro-ecosystems. Available data for validation of satellite-based remote sensing of N is scarce. Therefore, in this study, field spectrometer measurements were used to simulate data of the Sentinel-2 (S2) satellites developed for vegetation monitoring by the ESA. The prediction performance of normalized ratio indices (NRIs), random forest regression (RFR) and Gaussian processes regression (GPR) for plant-N-related traits was assessed on a diverse real-world dataset including multiple crops, field sites and years. The plant N traits included the mass-based N measure, N concentration in the biomass (Nconc), and an area-based N measure approximating the plant N uptake (NUP). Spectral indices such as normalized ratio indices (NRIs) performed well, but the RFR and GPR methods outperformed the NRIs. Key spectral bands for each trait were identified using the RFR variable importance measure and the Gaussian processes regression band analysis tool (GPR-BAT), highlighting the importance of the short-wave infrared (SWIR) region for estimation of plant Nconc-and to a lesser extent the NUP. The red edge (RE) region was also important. The GPR-BAT showed that five bands were sufficient for plant N trait and leaf area index (LAI) estimation and that a surplus of bands effectively reduced prediction performance. A global sensitivity analysis (GSA) was performed on all traits simultaneously, showing the dominance of the LAI in the mixed remote sensing signal. To delineate the plant-N-related traits from this signal, regional and/or national data collection campaigns producing large crop spectral libraries (CSL) are needed. An improved database will likely enable the mapping of N at the agro-ecosystem level or for use in precision farming by farmers in the future.

Systematic Assessment of MODTRAN Emulators for Atmospheric Correction

Atmospheric radiative transfer models (RTMs) simulate the light propagation in the Earth's atmosphere. With the evolution of RTMs, their increase in complexity makes them impractical in routine processing such as atmospheric correction. To overcome their computational burden, standard practice is to interpolate a multidimensional lookup table (LUT) of prestored simulations. However, accurate interpolation relies on large LUTs, which still implies large computation times for their generation and interpolation. In recent years, emulation has been proposed as an alternative to LUT interpolation. Emulation approximates the RTM outputs by a statistical regression model trained with a low number of RTM runs. However, a concern is whether the emulator reaches sufficient accuracy for atmospheric correction. Therefore, we have performed a systematic assessment of key aspects that impact the precision of emulating MODTRAN: 1) regression algorithm; 2) training database size; 3) dimensionality reduction (DR) method and a number of components; and 4) spectral resolution. The Gaussian processes regression (GPR) was found the most accurate emulator. The principal component analysis remains a robust DR method and nearly 20 components reach sufficient precision. Based on a database of 1000 samples covering a broad range of atmospheric conditions, GPR emulators can reconstruct the simulated spectral data with relative errors below 1% for the 95th percentile. These emulators reduce the processing time from days to minutes, preserving sufficient accuracy for atmospheric correction and providing model uncertainties and derivatives. We provide a set of guidelines and tools to design and generate accurate emulators for satellite data processing applications.

A Survey of Active Learning for Quantifying Vegetation Traits from Terrestrial Earth Observation Data

The current exponential increase of spatiotemporally explicit data streams from satellitebased Earth observation missions offers promising opportunities for global vegetation monitoring. Intelligent sampling through active learning (AL) heuristics provides a pathway for fast inference of essential vegetation variables by means of hybrid retrieval approaches, i.e., machine learning regression algorithms trained by radiative transfer model (RTM) simulations. In this study we summarize AL theory and perform a brief systematic literature survey about AL heuristics used in the context of Earth observation regression problems over terrestrial targets. Across all relevant studies it appeared that: (i) retrieval accuracy of AL-optimized training data sets outperformed models trained over large randomly sampled data sets, and (ii) Euclidean distance-based (EBD) diversity method tends to be the most efficient AL technique in terms of accuracy and computational demand. Additionally, a case study is presented based on experimental data employing both uncertainty and diversity AL criteria. Hereby, a a simulated training data base by the PROSAIL-PRO canopy RTM is used to demonstrate the benefit of AL techniques for the estimation of total leaf carotenoid content (Cxc ) and leaf water content (Cw ). Gaussian process regression (GPR) was incorporated to minimize and optimize the training data set with AL. Training the GPR algorithm on optimally AL-based sampled data sets led to improved variable retrievals compared to training on full data pools, which is further demonstrated on a mapping example. From these findings we can recommend the use of AL-based sub-sampling procedures to select the most informative samples out of large training data pools. This will not only optimize regression accuracy due to exclusion of redundant information, but also speed up processing time and reduce final model size of kernel-based machine learning regression algorithms, such as GPR. With this study we want to encourage further testing and implementation of AL sampling methods for hybrid retrieval workflows. AL can contribute to the solution of regression problems within the framework of operational vegetation monitoring using satellite imaging spectroscopy data, and may strongly facilitate data processing for cloud-computing platforms.

Partitioning net carbon dioxide fluxes into photosynthesis and respiration using neural networks

The eddy covariance (EC) technique is used to measure the net ecosystem exchange (NEE) of CO2 between ecosystems and the atmosphere, offering a unique opportunity to study ecosystem responses to climate change. NEE is the difference between the total CO2 release due to all respiration processes (RECO), and the gross carbon uptake by photosynthesis (GPP). These two gross CO2 fluxes are derived from EC measurements by applying partitioning methods that rely on physiologically based functional relationships with a limited number of environmental drivers. However, the partitioning methods applied in the global FLUXNET network of EC observations do not account for the multiple co-acting factors that modulate GPP and RECO flux dynamics. To overcome this limitation, we developed a hybrid data-driven approach based on combined neural networks (NNC-part ). NNC-part incorporates process knowledge by introducing a photosynthetic response based on the light-use efficiency (LUE) concept, and uses a comprehensive dataset of soil and micrometeorological variables as fluxes drivers. We applied the method to 36 sites from the FLUXNET2015 dataset and found a high consistency in the results with those derived from other standard partitioning methods for both GPP (R2  > .94) and RECO (R2  > .8). High consistency was also found for (a) the diurnal and seasonal patterns of fluxes and (b) the ecosystem functional responses. NNC-part performed more realistic than the traditional methods for predicting additional patterns of gross CO2 fluxes, such as: (a) the GPP response to VPD, (b) direct effects of air temperature on GPP dynamics, (c) hysteresis in the diel cycle of gross CO2 fluxes, (d) the sensitivity of LUE to the diffuse to direct radiation ratio, and (e) the post rain respiration pulse after a long dry period. In conclusion, NNC-part is a valid data-driven approach to provide GPP and RECO estimates and complementary to the existing partitioning methods.

Quantifying the Robustness of Vegetation Indices through Global Sensitivity Analysis of Homogeneous and Forest Leaf-Canopy Radiative Transfer Models

Vegetation indices (VIs) are widely used in optical remote sensing to estimate biophysical variables of vegetated surfaces. With the advent of spectroscopy technology, spectral bands can be combined in numerous ways to extract the desired information. This resulted in a plethora of proposed indices, designed for a diversity of applications and research purposes. However, it is not always clear whether they are sensitive to the variable of interest while at the same time, responding insensitive to confounding factors. Hence, to be able to quantify the robustness of VIs, a systematic evaluation is needed, thereby introducing a widest possible variety of biochemical and structural heterogeneity. Such exercise can be achieved with coupled leaf and canopy radiative transfer models (RTMs), whereby input variables can virtually simulate any vegetation scenario. With the intention of evaluating multiple VIs in an efficient way, this led us to the development of a global sensitivity analysis (GSA) toolbox dedicated to the analysis of VIs on their sensitivity towards RTM input variables. We identified VIs that are designed to be sensitive towards leaf chlorophyll content (LCC), leaf water content (LWC) and leaf area index (LAI) for common sensors of terrestrial Earth observation satellites: Landsat 8, MODIS, Sentinel-2, Sentinel-3 and the upcoming imaging spectrometer mission EnMAP. The coupled RTMs PROSAIL and PROINFORM were used for simulations of homogeneous and forest canopies respectively. GSA total sensitivity results suggest that LCC-sensitive indices respond most robust: for the great majority of scenarios, chlorophyll a + b content (Cab) drives between 75% and 82% of the indices' variability. LWC-sensitive indices were most affected by confounding variables such as Cab and LAI, although the equivalent water thickness (Cw) can drive between 25% and 50% of the indices' variability. Conversely, the majority of LAI-sensitive indices are not only sensitive to LAI but rather to a mixture of structural and biochemical variables.

Remote sensing of solar-induced chlorophyll fluorescence (SIF) in vegetation: 50 years of progress

Remote sensing of solar-induced chlorophyll fluorescence (SIF) is a rapidly advancing front in terrestrial vegetation science, with emerging capability in space-based methodologies and diverse application prospects. Although remote sensing of SIF - especially from space - is seen as a contemporary new specialty for terrestrial plants, it is founded upon a multi-decadal history of research, applications, and sensor developments in active and passive sensing of chlorophyll fluorescence. Current technical capabilities allow SIF to be measured across a range of biological, spatial, and temporal scales. As an optical signal, SIF may be assessed remotely using highly-resolved spectral sensors and state-of-the-art algorithms to distinguish the emission from reflected and/or scattered ambient light. Because the red to far-red SIF emission is detectable non-invasively, it may be sampled repeatedly to acquire spatio-temporally explicit information about photosynthetic light responses and steady-state behaviour in vegetation. Progress in this field is accelerating with innovative sensor developments, retrieval methods, and modelling advances. This review distills the historical and current developments spanning the last several decades. It highlights SIF heritage and complementarity within the broader field of fluorescence science, the maturation of physiological and radiative transfer modelling, SIF signal retrieval strategies, techniques for field and airborne sensing, advances in satellite-based systems, and applications of these capabilities in evaluation of photosynthesis and stress effects. Progress, challenges, and future directions are considered for this unique avenue of remote sensing.

Global Sensitivity Analysis of Leaf-Canopy-Atmosphere RTMs: Implications for Biophysical Variables Retrieval from Top-of-Atmosphere Radiance Data

Knowledge of key variables driving the top of the atmosphere (TOA) radiance over a vegetated surface is an important step to derive biophysical variables from TOA radiance data, e.g., as observed by an optical satellite. Coupled leaf-canopy-atmosphere Radiative Transfer Models (RTMs) allow linking vegetation variables directly to the at-sensor TOA radiance measured. Global Sensitivity Analysis (GSA) of RTMs enables the computation of the total contribution of each input variable to the output variance. We determined the impacts of the leaf-canopy-atmosphere variables into TOA radiance using the GSA to gain insights into retrievable variables. The leaf and canopy RTM PROSAIL was coupled with the atmospheric RTM MODTRAN5. Because of MODTRAN's computational burden and GSA's demand for many simulations, we first developed a surrogate statistical learning model, i.e., an emulator, that allows approximating RTM outputs through a machine learning algorithm with low computation time. A Gaussian process regression (GPR) emulator was used to reproduce lookup tables of TOA radiance as a function of 12 input variables with relative errors of 2.4%. GSA total sensitivity results quantified the driving variables of emulated TOA radiance along the 400-2500 nm spectral range at 15 cm-1 (between 0.3-9 nm); overall, the vegetation variables play a more dominant role than atmospheric variables. This suggests the possibility to retrieve biophysical variables directly from at-sensor TOA radiance data. Particularly promising are leaf chlorophyll content, leaf water thickness and leaf area index, as these variables are the most important drivers in governing TOA radiance outside the water absorption regions. A software framework was developed to facilitate the development of retrieval models from at-sensor TOA radiance data. As a proof of concept, maps of these biophysical variables have been generated for both TOA (L1C) and bottom-of-atmosphere (L2A) Sentinel-2 data by means of a hybrid retrieval scheme, i.e., training GPR retrieval algorithms using the RTM simulations. Obtained maps from L1C vs L2A data are consistent, suggesting that vegetation properties can be directly retrieved from TOA radiance data given a cloud-free sky, thus without the need of an atmospheric correction.

Gradient-Based Automatic Lookup Table Generator for Radiative Transfer Models

Physically based radiative transfer models (RTMs) are widely used in Earth observation to understand the radiation processes occurring on the Earth's surface and their interactions with water, vegetation, and atmosphere. Through continuous improvements, RTMs have increased in accuracy and representativity of complex scenes at expenses of an increase in complexity and computation time, making them impractical in various remote sensing applications. To overcome this limitation, the common practice is to precompute large lookup tables (LUTs) for their later interpolation. To further reduce the RTM computation burden and the error in LUT interpolation, we have developed a method to automatically select the minimum and optimal set of input-output points (nodes) to be included in an LUT. We present the gradient-based automatic LUT generator algorithm (GALGA), which relies on the notion of an acquisition function that incorporates: 1) the Jacobian evaluation of an RTM and 2) the information about the multivariate distribution of the current nodes. We illustrate the capabilities of GALGA in the automatic construction and optimization of MODTRAN-based LUTs of different dimensions of the input variables space. Our results indicate that when compared with a pseudorandom homogeneous distribution of the LUT nodes, GALGA reduces:1) the LUT size by >24%; 2) the computation time by 27%; and 3) the maximum interpolation relative errors by at least 10%. It is concluded that an automatic LUT design might benefit from the methodology proposed in GALGA to reduce interpolation errors and computation time in computationally expensive RTMs.

Approximating Empirical Surface Reflectance Data through Emulation: Opportunities for Synthetic Scene Generation

Collection of spectroradiometric measurements with associated biophysical variables is an essential part of the development and validation of optical remote sensing vegetation products. However, their quality can only be assessed in the subsequent analysis, and often there is a need for collecting extra data, e.g., to fill in gaps. To generate empirical-like surface reflectance data of vegetated surfaces, we propose to exploit emulation, i.e., reconstruction of spectral measurements through statistical learning. We evaluated emulation against classical interpolation methods using an empirical field dataset with associated hyperspectral spaceborne CHRIS and airborne HyMap reflectance spectra, to produce synthetic CHRIS and HyMap reflectance spectra for any combination of input biophysical variables. Results indicate that: (1) emulation produces surface reflectance data more accurately than interpolation when validating against a separate part of the field dataset; and (2) emulation produces the spectra multiple times (tens to hundreds) faster than interpolation. This technique opens various data processing opportunities, e.g., emulators not only allow rapidly producing large synthetic spectral datasets, but they can also speed up computationally intensive processing routines such as synthetic scene generation. To demonstrate this, emulators were run to simulate hyperspectral imagery based on input maps of a few biophysical variables coming from CHRIS, HyMap and Sentinel-2 (S2). The emulators produced spaceborne CHRIS-like and airborne HyMap-like surface reflectance imagery in the order of seconds, thereby approximating the spectra of vegetated surfaces sufficiently similar to the reference images. Similarly, it took a few minutes to produce a hyperspectral data cube with a spatial texture of S2 and a spectral resolution of HyMap.

Emulation as an Accurate Alternative to Interpolation in Sampling Radiative Transfer Codes

Computationally expensive radiative transfer models (RTMs) are widely used to realistically reproduce the light interaction with the earth surface and atmosphere. Because these models take long processing time, the common practice is to first generate a sparse look-up table (LUT) and then make use of interpolation methods to sample the multidimensional LUT input variable space. However, the question arise whether common interpolation methodsperform most accurate. As an alternative to interpolation, this paper proposes to use emulation, i.e., approximating the RTM output by means of the statistical learning. Two experiments were conducted to assess the accuracy in delivering spectral outputs using interpolation and emulation: at canopy level, using PROSAIL; and at top-of-atmosphere level, using MODTRAN. Various interpolation (nearest-neighbor, inverse distance weighting, and piece-wice linear) and emulation [Gaussian process regression (GPR), kernel ridge regression, and neural networks] methods were evaluated against a dense reference LUT. In all experiments, the emulation methods clearly produced more accurate output spectra than classical interpolation methods. The GPR emulation performed up to ten times more accurately than the best performing interpolation method, and this with a speed that is competitive with the faster interpolation methods. It is concluded that emulation can function as a fast and more accurate alternative to commonly used interpolation methods for reconstructing RTM spectral data.

Quantifying Vegetation Biophysical Variables from Imaging Spectroscopy Data: A Review on Retrieval Methods

An unprecedented spectroscopic data stream will soon become available with forthcoming Earth-observing satellite missions equipped with imaging spectroradiometers. This data stream will open up a vast array of opportunities to quantify a diversity of biochemical and structural vegetation properties. The processing requirements for such large data streams require reliable retrieval techniques enabling the spatiotemporally explicit quantification of biophysical variables. With the aim of preparing for this new era of Earth observation, this review summarizes the state-of-the-art retrieval methods that have been applied in experimental imaging spectroscopy studies inferring all kinds of vegetation biophysical variables. Identified retrieval methods are categorized into: (1) parametric regression, including vegetation indices, shape indices and spectral transformations; (2) nonparametric regression, including linear and nonlinear machine learning regression algorithms; (3) physically based, including inversion of radiative transfer models (RTMs) using numerical optimization and look-up table approaches; and (4) hybrid regression methods, which combine RTM simulations with machine learning regression methods. For each of these categories, an overview of widely applied methods with application to mapping vegetation properties is given. In view of processing imaging spectroscopy data, a critical aspect involves the challenge of dealing with spectral multicollinearity. The ability to provide robust estimates, retrieval uncertainties and acceptable retrieval processing speed are other important aspects in view of operational processing. Recommendations towards new-generation spectroscopy-based processing chains for operational production of biophysical variables are given.