1
|
Sun P, Hou M, Lyu S, Wang W, Li S, Mao J, Li S. Enhancement and Restoration of Scratched Murals Based on Hyperspectral Imaging-A Case Study of Murals in the Baoguang Hall of Qutan Temple, Qinghai, China. Sensors (Basel) 2022; 22:9780. [PMID: 36560152 PMCID: PMC9782035 DOI: 10.3390/s22249780] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Revised: 11/23/2022] [Accepted: 12/06/2022] [Indexed: 06/17/2023]
Abstract
Environmental changes and human activities have caused serious degradation of murals around the world. Scratches are one of the most common issues in these damaged murals. We propose a new method for virtually enhancing and removing scratches from murals; which can provide an auxiliary reference and support for actual restoration. First, principal component analysis (PCA) was performed on the hyperspectral data of a mural after reflectance correction, and high-pass filtering was performed on the selected first principal component image. Principal component fusion was used to replace the original first principal component with a high-pass filtered first principal component image, which was then inverse PCA transformed with the other original principal component images to obtain an enhanced hyperspectral image. The linear information in the mural was therefore enhanced, and the differences between the scratches and background improved. Second, the enhanced hyperspectral image of the mural was synthesized as a true colour image and converted to the HSV colour space. The light brightness component of the image was estimated using the multi-scale Gaussian function and corrected with a 2D gamma function, thus solving the problem of localised darkness in the murals. Finally, the enhanced mural images were applied as input to the triplet domain translation network pretrained model. The local branches in the translation network perform overall noise smoothing and colour recovery of the mural, while the partial nonlocal block is used to extract the information from the scratches. The mapping process was learned in the hidden space for virtual removal of the scratches. In addition, we added a Butterworth high-pass filter at the end of the network to generate the final restoration result of the mural with a clearer visual effect and richer high-frequency information. We verified and validated these methods for murals in the Baoguang Hall of Qutan Temple. The results show that the proposed method outperforms the restoration results of the total variation (TV) model, curvature-driven diffusion (CDD) model, and Criminisi algorithm. Moreover, the proposed combined method produces better recovery results and improves the visual richness, readability, and artistic expression of the murals compared with direct recovery using a triple domain translation network.
Collapse
Affiliation(s)
- Pengyu Sun
- School of Geomatics and Urban Spatial Informatics, Beijing University of Civil Engineering and Architecture, No. 15 Yongyuan Road, Beijing 102616, China
- Beijing Key Laboratory for Architectural Heritage Fine Reconstruction & Health Monitoring, No. 15 Yongyuan Road, Beijing 102616, China
| | - Miaole Hou
- School of Geomatics and Urban Spatial Informatics, Beijing University of Civil Engineering and Architecture, No. 15 Yongyuan Road, Beijing 102616, China
- Beijing Key Laboratory for Architectural Heritage Fine Reconstruction & Health Monitoring, No. 15 Yongyuan Road, Beijing 102616, China
| | - Shuqiang Lyu
- School of Geomatics and Urban Spatial Informatics, Beijing University of Civil Engineering and Architecture, No. 15 Yongyuan Road, Beijing 102616, China
- Beijing Key Laboratory for Architectural Heritage Fine Reconstruction & Health Monitoring, No. 15 Yongyuan Road, Beijing 102616, China
| | - Wanfu Wang
- The Dunhuang Academy, Dunhuang 736200, China
| | - Shuyang Li
- School of Geomatics and Urban Spatial Informatics, Beijing University of Civil Engineering and Architecture, No. 15 Yongyuan Road, Beijing 102616, China
- Beijing Key Laboratory for Architectural Heritage Fine Reconstruction & Health Monitoring, No. 15 Yongyuan Road, Beijing 102616, China
| | - Jincheng Mao
- School of Geomatics and Urban Spatial Informatics, Beijing University of Civil Engineering and Architecture, No. 15 Yongyuan Road, Beijing 102616, China
- Beijing Key Laboratory for Architectural Heritage Fine Reconstruction & Health Monitoring, No. 15 Yongyuan Road, Beijing 102616, China
| | - Songnian Li
- Department of Civil Engineering, Toronto Metropolitan University, 350 Victoria Street, Toronto, ON M5B 2K3, Canada
| |
Collapse
|