US-12626506-B2 - Method and system for detecting changes in areas

US12626506B2US 12626506 B2US12626506 B2US 12626506B2US-12626506-B2

Abstract

The present invention provides a method and system for detecting changes in areas, comprising: acquiring images at different times of an area by low-altitude UAVs; extracting state features for each of backbone feature pairs; analyzing importance of each of state parts due to the state parts have different importance degrees to the whole image features under different scenes; eliminating the state features from complete features by virtue of differences in importance degrees, to obtain corresponding essential parts; determining at least one change in the area by analyzing the essential parts. The invention comprehensively considers the importance of the state parts in the input images, and the essential parts contains complete and pure information are obtained, which reflects an actual changed region.

Inventors

Runmin CONG
Zifeng QIU
Wei Zhang
Ran Song
Liangbin ZHU
Yu Chen
Taoyi CHEN
XIAOLEI LI

Assignees

SHANDONG UNIVERSITY

Dates

Publication Date: 20260512
Application Date: 20241217
Priority Date: 20230804

Claims (7)

1 . A method for detecting changes in areas, comprising: acquiring images at different times of an area to be detected by using a low-altitude UAV; inputting the images acquired at different times into a trained change detection model to obtain a result of change detection of images of the area to be detected; and determining at least one changing feature of the area to be detected by analyzing the result of change detection of images of the area to be detected; wherein, the images acquired at different times of the area to be detected by using the low-altitude UAV, comprising: a pre-event image, obtained by shooting the area to be detected at a spatial point and a shooting angle by using the low-altitude UAV through an UAV control algorithm and a GPS positioning technology; wherein, the spatial point and the shooting angle is recorded; and, a post-event image, obtained by shooting the area to be detected again after a predetermined time interval, at the recorded spatial points and shooting angles by using the low-altitude UAV; and the inputting the images acquired at different times into the trained change detection model to obtain the result of change detection of images of the area to be detected, comprising: extracting backbone features from the images acquired at different times to obtain multilevel backbone feature pairs; extracting state features from the multilevel backbone feature pairs to obtain state feature pairs; obtaining state feature weights based on importance degrees of the state features in the images acquired at different times; and, obtaining essential feature pairs according to the state feature pairs, the backbone feature pairs, and the state feature weights corresponding to the state feature pairs respectively; and performing a cascade fusion on the essential feature pairs to obtain the result of change detection of images of the area to be detected; wherein, the extracting the state features from the multilevel backbone feature pairs to obtain the state feature pairs, and obtaining the state feature weights based on the importance degrees of the state features in the images acquired at different times, comprising: cascading the multilevel backbone feature pairs, and reducing a number of channels of the cascaded multilevel backbone feature pairs by passing through convolution layers, respectively, to obtain the state feature pairs; and adjusting a number of channels of the state features according to the importance degrees of the state features in the images acquired at different times, to obtain the state feature weights, specifically comprising: cascading each state in the state feature pairs with corresponding backbone feature; adjusting a number of channels of cascaded result by passing through a 3×3 convolution layers to be equal to the number of channels of the state feature pairs; adjusting again the number of channels of the cascaded result that after adjusting the number of channels of the cascaded result by sequentially passing through another two 3×3 convolution layers to be equal to the number of channels of the backbone feature pairs, and obtaining the state feature weights by performing a normalization processing through an activation function; the obtaining essential feature pairs according to the state feature pairs, the backbone feature pairs, and the state feature weights corresponding to the state feature pairs respectively, specifically comprising: performing a pixel-level multiplication operation on the state feature weights and corresponding state features to obtain a first result; performing a pixel-level subtraction operation on the first result and the backbone features corresponding to the state feature weights to obtain a second result; and performing a cascade operation on the second result and the backbone features corresponding to the state feature weights to obtain a third result, and obtaining the essential features by passing the third result through a 1×1 convolution layer.
2 . The method according to claim 1 , wherein the performing a cascade fusion on different the essential feature pairs to obtain the result of change detection of images of the area to be detected, specifically comprising: performing fusion on adjacent essential feature pairs layer by layer after upsampling the essential feature pairs, and obtaining the result of change detection of images of the area to be detected by passing a final fusion result through the 1×1 convolution layer.
3 . The method according to claim 1 , wherein a twin convolutional neural network composed of two VGG16 networks is used to extract the multilevel backbone feature pairs from the images acquired at different times.
4 . The method according to claim 1 , wherein a process of training a change detection model, comprising: acquiring background images and foreground regions clipped from images in a salient target detection dataset; pasting the foreground regions into the background images to forming image pairs with the background images; and training the change detection model by using a training set constructed from the image pairs and a dataset of the images acquired at different times.
5 . A system for detecting changes in area, comprising: a low-altitude UAV with an image capture apparatus, configured to: shoot a pre-event image and a post-event image of an area to be detected at different times and at a constant shooting angle at a constant space point based on an UAV control algorithm and a GPS positioning technology; forming images acquired at different times of the area to be detected by combining the pre-event image with the post-event image; a data-processing and changing-detection apparatus, configured to: process and input the images acquired at different times into a trained change detection model to obtain a result of change detection of images of the area to be detected; determining at least one changing feature of the area to be detected by analyzing the result of change detection of images of the area to be detected; wherein, the data-processing and changing-detection apparatus is configured to: extract backbone features from the images acquired at different times to obtain multilevel backbone feature pairs; extract state features from the multilevel backbone feature pairs to obtain state feature pairs; obtain state feature weights based on importance degrees of the state features in the images acquired at different times; and, obtain essential feature pairs according to the state feature pairs, the backbone feature pairs, and the state feature weights; and perform a cascade fusion on the essential feature pairs to obtain the result of change detection of images of the area to be detected; wherein, the extracting the state features from the multilevel backbone feature pairs to obtain the state feature pairs, and obtaining the state feature weights based on the importance degrees of the state features in the images acquired at different times, comprising: cascading the multilevel backbone feature pairs, and reducing a number of channels of the cascaded multilevel backbone feature pairs by passing through convolution layers, respectively, to obtain the state feature pairs; and adjusting a number of channels of the state features according to the importance degrees of the state features in the images acquired at different times, to obtain the state feature weights, specifically comprising: cascading each state in the state feature pairs with corresponding backbone feature; adjusting a number of channels of cascaded result by passing through a 3×3 convolution layers to be equal to the number of channels of the state feature pairs; adjusting again the number of channels of the cascaded result that after adjusting the number of channels of the cascaded result by sequentially passing through another two 3×3 convolution layers to be equal to the number of channels of the backbone feature pairs, and obtaining the state feature weights by performing a normalization processing through an activation function; the obtaining essential feature pairs according to the state feature pairs, the backbone feature pairs, and the state feature weights corresponding to the state feature pairs respectively, specifically comprising: performing a pixel-level multiplication operation on the state feature weights and corresponding state features to obtain a first result; performing a pixel-level subtraction operation on the first result and the backbone features corresponding to the state feature weights to obtain a second result; and performing a cascade operation on the second result and the backbone features corresponding to the state feature weights to obtain a third result, and obtaining the essential features by passing the third result through a 1×1 convolution layer.
6 . A computer device comprising: a processor, a memory, and a bus, wherein the memory storing machine-readable instructions executable by the processor, the processor communicating with the memory via the bus when the computer device is operating; wherein, when the machine-readable instructions are executed by the processor, implementing a method for detecting changes in areas according to claim 1 .
7 . A non-transitory computer-readable storage medium having a computer program stored thereon; wherein, when the computer program is executed by a processor, implementing a method for detecting changes in areas according to claim 1 .

Description

CROSS-REFERENCE TO RELATED APPLICATION This application is a continuation-in-part of international PCT application serial no. PCT/CN2024/086997, filed on Apr. 10, 2024, which claims priority benefit of China application no. 202310983540.5, filed on Aug. 4, 2023. The entirety of each of the above-mentioned patent applications is hereby incorporated by reference herein and made a part of this specification. TECHNICAL FIELD The present invention belongs to the technical field of computer vision, and particularly relates to a method and system for detecting changes in areas. BACKGROUND The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art. Change detection is a task aimed at locating and segmenting changing regions in pairs of images acquired at different times, and is important in many fields, such as urban supervision, disaster assessment and desert governance. Due to the wide application and great potential of this task, more and more methods have been proposed to solve the tasks of the change detection. In some early representative works, solve this task is solved by using traditional methods, while gradually improving performance. In recent years, deep learning (DL) models have been widely used in computer vision tasks, further stimulating researchers' interest in using the DL models to solve the tasks of the change detection. Since 2015, methods of change detection based on DL have evolved rapidly. Some existing methods introduce more innovative models, which consecutively refreshes the state-of-the-art performance of change detection or its subtasks. The data of change detection in images from low-altitude unmanned aerial vehicles (UAVs) are the images of street view taken by low-altitude imaging devices (such as drones), which play an important role in supporting the construction of smart cities. Compared with the task of change detection based on remote sensing, a diversity of objects and scenes in the change detection based on the low-altitude UAVs is higher, which means that more diverse and complex changes will occur, which is undoubtedly a more challenging task. In addition to meteorological conditions, the change detection based on the low-altitude UAVs also faces difficulties caused by obstruction from buildings, shadows of different shapes and depths, and complex and ever-changing artificial lighting. In order to achieve high quality detections, models must be able to identify essential changes and avoid the adverse effects described above as much as possible. Since each set of inputs consists of two images in the change detection based on the low-altitude UAVs, some of the previous work naturally applies twin convolutional neural networks to extract features. In a narrow sense, a twin convolutional neural network consists of two convolutional neural networks that share weights. When two input images respectively pass through each one of the two convolutional neural networks, features in the two images will be extracted in the same form. Ideally, the same regions are still similar in a feature map, and different regions may be represented as different features, making it easier to find changing regions between the two images (a pair of images) and meets the objective requirements of the task. However, all previous work using the twin convolutional neural networks has focused on how to correctly fuse feature pairs or amplify feature differences in changing regions, making them easier to locate. While these ideas help obtain higher quality change maps (actually, they do improve performance), but another key point has been overlooked, that is, finding changing regions in one task of change detection based on the low-altitude UAVs does not mean that all differences between the two images may be concerned. Usually, the changing regions to be found in the street view images taken by the low-altitude UAVs include two-dimensional (2D) changes in a surface of objects (such as murals on walls) and three-dimensional (3D) changes (such as the appearance, disappearance or movement of objects). It should be noted that changes caused by different lighting, shading, or color styles are considered to be different states of the same essence, i.e., “state changes.” Even if they are essentially the same, different states can lead to very large differences in representation, which can interfere with the model's prediction of the correct change map. Therefore, how to get rid of the influence of the state changes, only pay attention to the essential change, and improve the accuracy of the change detection based on the low-altitude UAVs is an urgent problem to be solved in the art. SUMMARY In order to overcome the shortcomings of the prior art, the present invention provides a method and system for detecting changes in areas, decoupling backbone features into essential parts and state parts, which makes a function unit of decodi