An Automatic Shadow Detection Method for VHR Remote Sensing Orthoimagery

Shadow Detection in Remote Sensing Images Based on Spectral Radiance Separability Enhancement

Shadow detection is a basic task of remote sensing image analysis, but it is often seriously disturbed by vegetation, water bodies, and black objects. It is observed that vegetation and dark objects often show a dark look in visible bands but brighter in the near-infrared (NIR), and is also noticed that the reflection of inland water bodies in the green band is stronger than that in the blue band. Taking advantage of these physical properties and combining them with the bluish and dark appearance of shadows, we propose a simple but effective shadow detection method for multispectral remote sensing images. These physical properties are used to create transformation models that suppress features such as vegetation, water bodies, etc., but at the same time enhance shadows. Then, we transform the shadow representation into a color space to generate candidate shadows using dominant color components. To separate shadows from the others, we propose two indexes, the normalized Color Difference Composite Index (CDCI) and Color Purity Index (CPI), and fuse them to achieve shadows and their confidence. The experimental results indicate that the proposed method can effectively detect the shadows in multispectral images and outperforms the state-of-the-art approaches.

AP Shadow Net: A Remote Sensing Shadow Removal Network Based on Atmospheric Transport and Poisson's Equation

Shadow is one of the fundamental indicators of remote sensing image which could cause loss or interference of the target data. As a result, the detection and removal of shadow has already been the hotspot of current study because of the complicated background information. In the following passage, a model combining the Atmospheric Transport Model (hereinafter abbreviated as ATM) with the Poisson Equation, AP ShadowNet, is proposed for the shadow detection and removal of remote sensing images by unsupervised learning. This network based on a preprocessing network based on ATM, A Net, and a network based on the Poisson Equation, P Net. Firstly, corresponding mapping between shadow and unshaded area is generated by the ATM. The brightened image will then enter the Confrontation identification in the P Net. Lastly, the reconstructed image is optimized on color consistency and edge transition by Poisson Equation. At present, most shadow removal models based on neural networks are significantly data-driven. Fortunately, by the model in this passage, the unsupervised shadow detection and removal could be released from the data source restrictions from the remote sensing images themselves. By verifying the shadow removal on our model, the result shows a satisfying effect from a both qualitative and quantitative angle. From a qualitative point of view, our results have a prominent effect on tone consistency and removal of detailed shadows. From the quantitative point of view, we adopt the non-reference evaluation indicators: gradient structure similarity (NRSS) and Natural Image Quality Evaluator (NIQE). Combining various evaluation factors such as reasoning speed and memory occupation, it shows that it is outstanding among other current algorithms.

Post-Processing for Shadow Detection in Drone-Acquired Images Using U-NET

Shadows in drone images commonly appear in various shapes, sizes, and brightness levels, as the images capture a wide view of scenery under many conditions, such as varied flying height and weather. This property of drone images leads to a major problem when it comes to detecting shadow and causes the presence of noise in the predicted shadow mask. The purpose of this study is to improve shadow detection results by implementing post-processing methods related to automatic thresholding and binary mask refinement. The aim is to discuss how the selected automatic thresholding and two methods of binary mask refinement perform to increase the efficiency and accuracy of shadow detection. The selected automatic thresholding method is Otsu’s thresholding, and methods for binary mask refinement are morphological operation and dense CRF. The study shows that the proposed methods achieve an acceptable accuracy of 96.43%.

Adaptive Unsupervised-Shadow-Detection Approach for Remote-Sensing Image Based on Multichannel Features

Shadow detection is an essential research topic in the remote-sensing domain, as the presence of shadow causes the loss of ground-object information in real areas. It is hard to define specific threshold values for the identification of shadow areas with the existing unsupervised approaches due to the complexity of remote-sensing scenes. In this study, an adaptive unsupervised-shadow-detection method based on multichannel features is proposed, which can adaptively distinguish shadow in different scenes. First, new multichannel features were designed in the hue, saturation, and intensity color space, and the shadow properties of high hue, high saturation, and low intensity were considered to solve the insufficient feature-extraction problem of shadows. Then, a dynamic local adaptive particle swarm optimization was proposed to calculate the segmentation thresholds for shadows in an adaptive manner. Finally, experiments performed on the Aerial Imagery dataset for Shadow Detection (AISD) demonstrated the superior performance of the proposed approach in comparison with traditional unsupervised shadow-detection and state-of-the-art deep-learning methods. The experimental results show that the proposed approach can detect the shadow areas in remote-sensing images more accurately and efficiently, with the F index being 82.70% on the testing images. Thus, the proposed approach has better application potential in scenarios without a large number of labeled samples.

Panoptic Segmentation Meets Remote Sensing

Panoptic segmentation combines instance and semantic predictions, allowing the detection of countable objects and different backgrounds simultaneously. Effectively approaching panoptic segmentation in remotely sensed data is very promising since it provides a complete classification, especially in areas with many elements as the urban setting. However, some difficulties have prevented the growth of this task: (a) it is very laborious to label large images with many classes, (b) there is no software for generating DL samples in the panoptic segmentation format, (c) remote sensing images are often very large requiring methods for selecting and generating samples, and (d) most available software is not friendly to remote sensing data formats (e.g., TIFF). Thus, this study aims to increase the operability of panoptic segmentation in remote sensing by providing: (1) a pipeline for generating panoptic segmentation datasets, (2) software to create deep learning samples in the Common Objects in Context (COCO) annotation format automatically, (3) a novel dataset, (4) leverage the Detectron2 software for compatibility with remote sensing data, and (5) evaluate this task on the urban setting. The proposed pipeline considers three inputs (original image, semantic image, and panoptic image), and our software uses these inputs alongside point shapefiles to automatically generate samples in the COCO annotation format. We generated 3400 samples with 512 × 512 pixel dimensions and evaluated the dataset using Panoptic-FPN. Besides, the metric analysis considered semantic, instance, and panoptic metrics, obtaining 93.865 mean intersection over union (mIoU), 47.691 Average (AP) Precision, and 64.979 Panoptic Quality (PQ). Our study presents the first effective pipeline for generating panoptic segmentation data for remote sensing targets.

Study on Local to Global Radiometric Balance for Remotely Sensed Imagery

Due to the difference of factors, such as lighting conditions, shooting environments, and time, there is compound brightness difference between adjacent images, which includes local brightness difference and radiometric difference. This paper proposed a method to eliminate the compound brightness difference of adjacent images after mosaicking, named local to global radiometric balance. It includes the brightness compensation model and brightness approach model. Firstly, the weighted average value of each row and column of image are calculated to express the brightness change; secondly, according to weighted average value, the brightness compensation model is built; thirdly, combined with the blocking method, the brightness compensation model is applied to image. Based on the value after above process, the brightness approach model is established to make the gray value of adjacent images approach to the mosaic line. In the paper, the standard deviation, MSE (mean square error) and mean value are used as the measure indices of the effect of brightness balance. The three groups of experimental results show that compared with the brightness stretch method, the histogram equalization method and the radiometric balance method, the local to global radiometric balance method not only realizes compound brightness balance, but also has better visual effects than others.

AgroShadow: A New Sentinel-2 Cloud Shadow Detection Tool for Precision Agriculture

Remote sensing for precision agriculture has been strongly fostered by the launches of the European Space Agency Sentinel-2 optical imaging constellation, enabling both academic and private services for redirecting farmers towards a more productive and sustainable management of the agroecosystems. As well as the freely and open access policy adopted by the European Space Agency (ESA), software and tools are also available for data processing and deeper analysis. Nowadays, a bottleneck in this valuable chain is represented by the difficulty in shadow identification of Sentinel-2 data that, for precision agriculture applications, results in a tedious problem. To overcome the issue, we present a simplified tool, AgroShadow, to gain full advantage from Sentinel-2 products and solve the trade-off between omission errors of Sen2Cor (the algorithm used by the ESA) and commission errors of MAJA (the algorithm used by Centre National d’Etudes Spatiales/Deutsches Zentrum für Luft- und Raumfahrt, CNES/DLR). AgroShadow was tested and compared against Sen2Cor and MAJA in 33 Sentinel 2A-B scenes, covering the whole of 2020 and in 18 different scenarios of the whole Italian country at farming scale. AgroShadow returned the lowest error and the highest accuracy and F-score, while precision, recall, specificity, and false positive rates were always similar to the best scores which alternately were returned by Sen2Cor or MAJA.

The Influence of Shadow Effects on the Spectral Characteristics of Glacial Meltwater

The phenomenon of shadows due to glaciers is investigated in Antarctica. The observed shadow effect disrupts analyses conducted by remote sensing and is a challenge in the assessment of sediment meltwater plumes in polar marine environments. A DJI Inspire 2 drone equipped with a Zenmuse x5s camera was used to generate a digital surface model (DSM) of 6 King George Island glaciers: Ecology, Dera, Zalewski, Ladies, Krak, and Vieville. On this basis, shaded areas of coves near glaciers were traced. For the first time, spectral characteristics of shaded meltwater were observed with the simultaneous use of a Sequoia+ spectral camera mounted on a Parrot Bluegrass drone and in Landsat 8 satellite images. In total, 44 drone flights were made, and 399 satellite images were analyzed. Among them, four drone spectral images and four satellite images were selected, meeting the condition of a visible shadow. For homogeneous waters (deep, low turbidity, without ice phenomena), the spectral properties tend to change during the approach to an obstacle casting a shadow especially during low shortwave downward radiation. In this case, in the shade, the amount of radiation reflected in the green spectral band decreases by 50% far from the obstacle and by 43% near the obstacle, while in near infrared (NIR), it decreases by 42% and 21%, respectively. With highly turbid, shallow water and ice phenomena, this tendency does not occur. It was found that the green spectral band had the highest contrast in the amount of reflected radiation between nonshaded and shaded areas, but due to its high sensitivity, the analysis could have been overestimated. The spectral properties of shaded meltwater differ depending on the distance from the glacier front, which is related to the saturation of the water with sediment particles. We discovered that the pixel aggregation of uniform areas caused the loss of detailed information, while pixel aggregation of nonuniform, shallow areas with ice phenomena caused changes and the loss of original information. During the aggregation of the original pixel resolution (15 cm) up to 30 m, the smallest error occurred in the area with a homogeneous water surface, while the greatest error (over 100%) was identified in the places where the water was strongly cloudy or there were ice phenomena.

Irradiance Restoration Based Shadow Compensation Approach for High Resolution Multispectral Satellite Remote Sensing Images

Numerous applications are hindered by shadows in high resolution satellite remote sensing images, like image classification, target recognition and change detection. In order to improve remote sensing image utilization, significant importance appears for restoring surface feature information under shadow regions. Problems inevitably occur for current shadow compensation methods in processing high resolution multispectral satellite remote sensing images, such as color distortion of compensated shadow and interference of non-shadow. In this study, to further settle these problems, we analyzed the surface irradiance of both shadow and non-shadow areas based on a satellite sensor imaging mechanism and radiative transfer theory, and finally develop an irradiance restoration based (IRB) shadow compensation approach under the assumption that the shadow area owns the same irradiance to the nearby non-shadow area containing the same type features. To validate the performance of the proposed IRB approach for shadow compensation, we tested numerous images of WorldView-2 and WorldView-3 acquired at different sites and times. We particularly evaluated the shadow compensation performance of the proposed IRB approach by qualitative visual sense comparison and quantitative assessment with two WorldView-3 test images of Tripoli, Libya. The resulting images automatically produced by our IRB method deliver a good visual sense and relatively low relative root mean square error (rRMSE) values. Experimental results show that the proposed IRB shadow compensation approach can not only compensate information of surface features in shadow areas both effectively and automatically, but can also well preserve information of objects in non-shadow regions for high resolution multispectral satellite remote sensing images.

GSCA-UNet: Towards Automatic Shadow Detection in Urban Aerial Imagery with Global-Spatial-Context Attention Module

As an inevitable phenomenon in most optical remote-sensing images, the effect of shadows is prominent in urban scenes. Shadow detection is critical for exploiting shadows and recovering the distorted information. Unfortunately, in general, automatic shadow detection methods for urban aerial images cannot achieve satisfactory performance due to the limitation of feature patterns and the lack of consideration of non-local contextual information. To address this challenging problem, the global-spatial-context-attention (GSCA) module was developed to self-adaptively aggregate all global contextual information over the spatial dimension for each pixel in this paper. The GSCA module was embedded into a modified U-shaped encoder–decoder network that was derived from the UNet network to output the final shadow predictions. The network was trained on a newly created shadow detection dataset, and the binary cross-entropy (BCE) loss function was modified to enhance the training procedure. The performance of the proposed method was evaluated on several typical urban aerial images. Experiment results suggested that the proposed method achieved a better trade-off between automaticity and accuracy. The F1-score, overall accuracy, balanced-error-rate, and intersection-over-union metrics of the proposed method were higher than those of other state-of-the-art shadow detection methods.

Building Shadow Detection on Ghost Images

Although many efforts have been made on building shadow detection from aerial images, little research on simultaneous shadows detection on both building roofs and grounds has been presented. Hence, this paper proposes a new method for simultaneous shadow detection on ghost image. In the proposed method, a corner point on shadow boundary is selected and its 3D approximate coordinate is calculated through photogrammetric collinear equation on the basis of assumption of average elevation within the aerial image. The 3D coordinates of the shadow corner point on shadow boundary is used to calculate the solar zenith angle and the solar altitude angle. The shadow areas on the ground, at the moment of aerial photograph shooting are determined by the solar zenith angle and the solar altitude angle with the prior information of the digital building model (DBM). Using the relationship between the shadows of each building and the height difference of buildings, whether there exists a shadow on the building roof is determined, and the shadow area on the building roof on the ghost image is detected on the basis of the DBM. High-resolution aerial images located in the City of Denver, Colorado, USA are used to verify the proposed method. The experimental results demonstrate that the shadows of the 120 buildings in the study area are completely detected, and the success rate is 15% higher than the traditional shadow detection method based on shadow features. Especially, when the shadows occur on the ground and on the buildings roofs, the successful rate of shadow detection can be improved by 9.42% and 33.33% respectively.

Hyperspectral analysis of objects under shadow conditions based on field reflectance measurements

Shaded objects in remote sensing data are usually neglected because of their low reflectance. The common treatment for detecting shadows consists of simple identification and straightforward removal; however, this approach is often criticized owing to the complexity and difficulty of its application and the incompleteness of the treated data. Recent efforts with hyperspectral field reflectance measurements have provided alternatives to scrutinize the issues of shadows. To demonstrate our proposed framework, we first gathered basic hyperspectral data with a spectrometer at Peking University, Beijing, China. Subsequently, we classified the spectral reflectances into three types: sunlight reflectance, shadow reflectance, and simulated reflectance. The spectral characteristics of shadows can provide pertinent classification information despite their low reflectance. Under full shadow conditions, the simulated reflectance was close to the sunlight reflectance because the same reduction ratio was used for the spectralon panel during the measurement process. In addition, the ratio of the shadow reflectance to the sunlight reflectance seemed to be related to the characteristics of the obstruction. The findings in this paper can contribute to a comprehensive understanding of the influence of shadows in remote sensing research, demonstrating the potential of fully utilizing typically neglected shadow information.

Extraction of Urban Objects in Cloud Shadows on the basis of Fusion of Airborne LiDAR and Hyperspectral Data

Feature extraction in cloud shadows is a difficult problem in the field of optical remote sensing. The key to solving this problem is to improve the accuracy of classification algorithms by fusing multi-source remotely sensed data. Hyperspectral data have rich spectral information but highly suffer from cloud shadows, whereas light detection and ranging (LiDAR) data can be acquired from beneath clouds to provide accurate height information. In this study, fused airborne LiDAR and hyperspectral data were used to extract urban objects in cloud shadows using the following steps: (1) a series of LiDAR and hyperspectral metrics were extracted and selected; (2) cloud shadows were extracted; (3) the new proposed approach was used by combining a pixel-based support vector machine (SVM) and object-based classifiers to extract urban objects in cloud shadows; (4) a pixel-based SVM classifier was used for the classification of the whole study area with the selected metrics; (5) a decision-fusion strategy was employed to get the final results for the whole study area; (6) accuracy assessment was conducted. Compared with the SVM classification results, the decision-fusion results of the combined SVM and object-based classifiers show that the overall classification accuracy is improved by 5.00% (from 87.30% to 92.30%). The experimental results confirm that the proposed method is very effective for urban object extraction in cloud shadows and thus improve urban applications such as urban green land management, land use analysis, and impervious surface assessment.

Automatic Shadow Detection in Urban Very-High-Resolution Images Using Existing 3D Models for Free Training

Up-to-date 3D city models are needed for many applications. Very-high-resolution (VHR) images with rich geometric and spectral information and a high update rate are increasingly applied for the purpose of updating 3D models. Shadow detection is the primary step for image interpretation, as shadow causes radiometric distortions. In addition, shadow itself is valuable geometric information. However, shadows are often complicated and environment-dependent. Supervised learning is considered to perform well in detecting shadows when training samples selected from these images are available. Unfortunately, manual labeling of images is expensive. Existing 3D models have been used to reconstruct shadows to provide free, computer-generated training samples, i.e., samples free from intensive manual labeling. However, accurate shadow reconstruction for large 3D models consisting of millions of triangles is either difficult or time-consuming. In addition, due to inaccuracy and incompleteness of the model, and different acquisition time between 3D models and images, mislabeling refers to training samples that are shadows but labeled as non-shadows and vice versa. We propose a ray-tracing approach with an effective KD tree construction to feasibly reconstruct accurate shadows for a large 3D model. An adaptive erosion approach is first provided to remove mislabeling effects near shadow boundaries. Next, a comparative study considering four classification methods, quadratic discriminant analysis (QDA) fusion, support vector machine (SVM), K nearest neighbors (KNN) and Random forest (RF), is performed to select the best classification method with respect to capturing the complicated properties of shadows and robustness to mislabeling. The experiments are performed on Dutch Amersfoort data with around 20% mislabels and the Toronto benchmark by simulating mislabels from inverting shadows to non-shadows. RF is tested to give robust and best results with 95.38% overall accuracy (OA) and a value of 0.9 for kappa coefficient (KC) for Amersfoort and around 96% OA and 0.92 KC for Toronto benchmarks when no more than 50% of shadows are inverted. QDA fusion and KNN are tested to be robust to mislabels but their capability to capture complicated properties of shadows is worse than RF. SVM is tested to have a good capability to separate shadow and non-shadows but is largely affected by mislabeled samples. It is shown that RF with free-training samples from existing 3D models is an automatic, effective, and robust approach for shadow detection from VHR images.

A Mixed Property-Based Automatic Shadow Detection Approach for VHR Multispectral Remote Sensing Images

Shadows in very high-resolution multispectral remote sensing images hinder many applications, such as change detection, target recognition, and image classification. Though a wide variety of significant research has explored shadow detection, shadow pixels are still more or less omitted and are wrongly confused with vegetation pixels in some cases. In this study, to further manage the problems of shadow omission and vegetation misclassification, a mixed property-based shadow index is developed for detecting shadows in very high-resolution multispectral remote sensing images based on the difference of the hue component and the intensity component between shadows and nonshadows, and the difference of the reflectivity of the red band and the near infrared band between shadows and vegetation cover in nonshadows. Then, the final shadow mask is achieved, with an optimal threshold automatically obtained from the index image histogram. To validate the effectiveness of our approach for shadow detection, three test images are selected from the multispectral WorldView-3 images of Rio de Janeiro, Brazil, and are tested with our method. When compared with other investigated standard shadow detection methods, the resulting images produced by our method deliver a higher average overall accuracy (95.02%) and a better visual sense. The highly accurate data show the efficacy and stability of the proposed approach in appropriately detecting shadows and correctly classifying shadow pixels against the vegetation pixels for very high-resolution multispectral remote sensing images.