CN-122024001-A - Building damage detection method and system for optical image and SAR image

CN122024001ACN 122024001 ACN122024001 ACN 122024001ACN-122024001-A

Abstract

The invention discloses a building damage detection method of an optical image and an SAR image, which comprises the steps of obtaining the optical image and the SAR image; the building damage detection model training method comprises the steps of manufacturing a data set of building damage of a heterogeneous image, building a heterogeneous change detection deep learning network, a decoding part, and training the heterogeneous change detection deep learning network by utilizing the building damage data set to obtain the building damage detection model, wherein the coding part comprises a structure guiding coding module with a built-in key point detection module, the structure guiding coding module is used for extracting high-level semantic information of the heterogeneous image and outputting a corresponding deep feature map, and the decoding part is used for merging the high-level semantic information of the heterogeneous image and extracting semantic change in the corresponding deep feature map through a semantic change extraction module. Based on the method, the precision, the reliability and the stability of building damage detection are improved.

Inventors

MA GUORUI
WANG DI
WU MAOMAO
YANG SHUO
ZHANG YINGJIE
QIN XIAOWEI

Assignees

武汉大学
北京机电工程研究所

Dates

Publication Date: 20260512
Application Date: 20251212
Priority Date: 20251128

Claims (10)

1. The method for detecting the building damage of the optical image and the SAR image is characterized by comprising the following steps of: Acquiring an optical image and an SAR image; inputting the optical image and the SAR image into a trained building damage detection model, and outputting a building damage detection result, wherein the training of the building damage detection model comprises the following steps: manufacturing a data set of the building damage of the heterogeneous image; the method comprises the steps of constructing a heterogeneous change detection deep learning network, wherein the coding part comprises a plurality of structure guiding coding modules, each structure guiding coding module is provided with a key point detection module, extracting high-level semantic information of an optical image and an SAR image through the plurality of structure guiding coding modules, and outputting a corresponding deep feature map; the decoding part comprises a semantic change extraction module, and combines the advanced semantic information of the optical image and the SAR image through the semantic change extraction module, and extracts semantic change in a deep feature map; training the heterogeneous change detection deep learning network by utilizing the data set of the heterogeneous image building damage to obtain a building damage detection model.
2. The method for detecting building damage of optical image and SAR image according to claim 1, wherein in each structure-guided encoding module, extracting key point features of the optical image and SAR image by the key point detection module comprises: Compressing the characteristics of the characteristic diagram in the horizontal and vertical directions by using an average pooling method to obtain position channel information in two directions; Weighting and reclassifying the position channel information in two directions by using two SE channel attention mechanisms respectively, and extracting the position characteristics in the two directions through convolution operation; Multiplying the extracted position features in two directions to obtain complete position features; the complete location feature is mapped onto the original feature map to obtain a feature map containing key point location features.
3. The method for detecting building damage of optical image and SAR image according to claim 1, wherein extracting semantic changes in the deep feature map comprises: deforming the depth feature map; calculating the similarity of each channel according to the matrix dot product to obtain a similarity graph; obtaining difference information among channels according to the maximum value information in the similarity graph; The difference information between channels is mapped to the original depth feature map to obtain a semantic change map.
4. The method for detecting building damage to optical image and SAR image according to claim 1, wherein said creating a data set of building damage to said heterologous image comprises: According to the texture and color characteristics of the optical image and the SAR image, building damage boundary sketching is carried out on the optical image and the SAR image by utilizing ArcGIS software, and a sketched image set is obtained; Sequentially cutting and dividing the sketched image set into a plurality of images according to preset sizes, taking a building damage area in the images as a positive sample, and taking a non-building damage area in the images as a negative sample; the image set is divided into a training set and a verification set.
5. The method for detecting building damage of optical image and SAR image according to claim 1, wherein after the optical image and SAR image are acquired, the SAR image is preprocessed, and the optical image and the preprocessed SAR image are subjected to image registration, wherein the preprocessing includes data import, multiview processing, filter processing, and geo-coding.
6. The method for detecting building damage of optical image and SAR image as set forth in claim 5, wherein SAR-SIFT algorithm is adopted to perform image registration on the optical image and the preprocessed SAR image.
7. The method for detecting the building damage of the optical image and the SAR image according to claim 1, wherein the building damage detection result output by the building damage detection model is quantitatively evaluated by adopting four evaluation indexes of overall precision, average precision, error early warning and overall error, and the four evaluation indexes are expressed as follows: , , , , wherein OA represents overall accuracy, AA represents average accuracy, FA represents error early warning, TE represents overall error, TP represents the number of positive classes correctly predicted by the model, TN represents the number of negative classes correctly predicted by the model, FN represents the number of negative classes incorrectly predicted by the model, and FP represents the number of positive classes incorrectly predicted by the model.
8. A building damage detection system for optical and SAR images, comprising: The image acquisition module is used for acquiring a pre-disaster optical image and a post-disaster SAR image; The building damage detection module inputs the pre-disaster optical image and the post-disaster SAR image into a trained building damage detection model and outputs a building damage detection result, wherein the training of the building damage detection model comprises the following steps: wherein, training of the building damage detection model comprises: manufacturing a data set of the building damage of the heterogeneous image; the method comprises the steps of constructing a heterogeneous change detection deep learning network, wherein the coding part comprises a plurality of structure guiding coding modules, each structure guiding coding module is provided with a key point detection module, extracting high-level semantic information of an optical image and an SAR image through the plurality of structure guiding coding modules, and outputting a corresponding deep feature map; the decoding part comprises a semantic change extraction module, and combines the advanced semantic information of the optical image and the SAR image through the semantic change extraction module, and extracts semantic change in a deep feature map; training the heterogeneous change detection deep learning network by utilizing the data set of the heterogeneous image building damage to obtain a building damage detection model.
9. An electronic device comprising a memory and a processor, the memory storing program instructions for execution by the processor, the processor invoking the program instructions to perform a method of building damage detection for an optical image and SAR image according to any of claims 1-7.
10. A non-transitory computer readable storage medium storing computer instructions that cause the computer to perform a method of building damage detection of an optical image and SAR image according to any one of claims 1 to 7.

Description

Building damage detection method and system for optical image and SAR image Technical Field The invention relates to the technical field of remote sensing image processing and computer vision, in particular to a building damage detection method and system for optical images and SAR images. Background Building damage assessment is a key index of war and conflict, and can provide decision basis and technical support for post-war rescue, decision command and the like. Therefore, how to timely and accurately obtain the post-war building damage information is a problem to be solved urgently. "Real-time" and "accuracy" are the general directions of building damage assessment development. The comprehensive, real-time and accurate acquisition of battlefield information is a foundation and a premise for realizing development. But a battlefield situation that is too complex is a great challenge for battlefield information acquisition. On one hand, the real-time performance of information acquisition is difficult to guarantee. The current means for acquiring image information mainly comprise remote sensing, unmanned aerial vehicle, ground monitoring equipment and the like. The remote sensing image has the advantages of high spatial resolution, large range and large information quantity, but the revisitation period is too long and is difficult to realize in real time. Although the unmanned aerial vehicle can realize real-time monitoring, the unmanned aerial vehicle is not stable enough and is easy to be damaged by accident to fail. The ground monitoring equipment can also realize real-time monitoring and is stable, but the angle is fixed, the range is too small, and the global overview requirement cannot be met. In modern battlefield, obtaining real-time image information with long-term stability, large range and high resolution becomes a technical problem to be solved urgently. On the other hand, the accuracy and availability of information acquisition is very important. The availability of information does not mean that the information is available and that the large amounts of smoke, light, fire and weapons equipment that are generated after a battle may affect the availability and accuracy of the acquired image information. The accuracy and usability of the obtained image information are enhanced by carrying out correlation processing on the obtained image information, and the timeliness of the information can be affected. With the continuous development of remote sensing technology, the extraction of post-war building damage information by utilizing remote sensing data has become an important approach. The building damage detection based on the high-resolution remote sensing image becomes a research hot spot for post-war evaluation in recent years due to abundant spectral characteristics and texture information. However, it is difficult to obtain a large amount of accurate and usable real-time image information, and especially, the optical image cannot be fully adapted to a series of special environments after the war, so that the damage evaluation technology based on the optical image completely has limitations in terms of 'real-time' and 'accuracy'. Synthetic Aperture Radar (SAR) is an active imaging mode, and can effectively avoid the influence of severe environment on observation due to the characteristics of full time, all weather and strong penetrating power. And a plurality of researchers analyze the polarization information and the coherent characteristic change before and after SAR images, so that the aim of building damage detection is fulfilled. However, these methods have relatively high requirements on data quality and are greatly affected by other features, so that the method for combining multi-source data such as optical images and SAR images has become a current research trend, and the advantages of each sensor can be effectively exerted. The SAR image based on Sentinel-1 and the optical image of Sentinel-2, putri et al complete the detection of building damages by adopting a random forest method, and the evaluation of building damages is completed by adopting a multi-source data fusion and multiple machine learning model integration method based on ALOS-2, PALSAR-2, the SAR image of Sentinel-1 and the optical images of Sentinel-2 and Planet Scope, bruno Adriano et al. The traditional machine learning algorithm is adopted for information extraction, so that semantic information in the optical image and SAR image cannot be deeply mined. From the above, it can be seen that the method for detecting the homologous variation in the heterogeneous building damage by using the optical image and the SAR image is a challenging task, and the existing method for detecting the homologous variation is directly applied to the heterogeneous building damage detection, which faces the problems of noise interference of the SAR image, different domains of the characteristic space of the heterogeneous image, and the l