CN-122023557-A - Method for generating visible light image weather diversity of unmanned aerial vehicle based on generation countermeasure network

CN122023557ACN 122023557 ACN122023557 ACN 122023557ACN-122023557-A

Abstract

The invention belongs to the technical field of image generation, and discloses a visible light image weather diversity generation method based on a generation countermeasure network unmanned aerial vehicle. The invention provides a method for generating visible light image weather diversity of an unmanned aerial vehicle based on a multi-domain image conversion network StarGAN, wherein a least square loss and an attention mechanism are introduced, the image fidelity of the generated unmanned aerial vehicle is improved, meanwhile, the image diversity of the unmanned aerial vehicle is enriched, the least square loss can effectively relieve the problems of instability, mode collapse and the like in the training process, the method is beneficial to generating a clearer and more real image, and the attention mechanism can effectively make up the defect of the generated image on background consistency, so that the network is more focused on a target area of the unmanned aerial vehicle.

Inventors

LONG ZHAOXIN
GAN CHUNQUAN
FAN HONGYING
WANG XUN
MENG QINGAN
JIANG ZEWEI
LIU XUDONG
ZHAO QIAN
ZOU KANG

Assignees

西南技术物理研究所

Dates

Publication Date: 20260512
Application Date: 20251224

Claims (10)

1. The method for generating the weather diversity of the visible light image of the unmanned aerial vehicle based on the generation countermeasure network is characterized by comprising a training process and an reasoning process, wherein the training process is performed by using images in a training data set, and the reasoning process is performed by using images in a test data set.
2. The method for generating weather diversity based on generating visible images of an opposing network drone of claim 1, wherein the training process comprises the steps of: s1, acquiring an unmanned aerial vehicle image generated by computer modeling, performing preprocessing operation, embedding an unmanned aerial vehicle region into a real background image in a training data set, adjusting the size of the image to 256×256 resolution, and recording the obtained image as an unmanned aerial vehicle physical simulation image; S2, inputting an unmanned aerial vehicle physical simulation image in a source domain and a target domain style code to a generator, wherein the target domain style refers to the style in a real background image, and encoding by using One-Hot; s3, the generator receives the unmanned aerial vehicle physical simulation image and the target domain style code as input, and outputs a generated image of the unmanned aerial vehicle target domain style and a mask image 1 of the unmanned aerial vehicle target area; s4, multiplying the generated image and the mask image 1 to obtain a foreground image, and multiplying the input simulation image and the inverted mask image 1 to obtain a background image; S5, adding a foreground image representing the target area of the unmanned aerial vehicle and a background image outside the target area of the unmanned aerial vehicle to obtain a synthetic image, wherein the synthetic image represents a finally generated target area style image of the unmanned aerial vehicle, the background is consistent with an input physical simulation image of the unmanned aerial vehicle, and the background consistency of the synthetic image and the input image is enhanced; S6, inputting a synthesized image of a target domain style of the unmanned aerial vehicle and a source domain style code to a generator, wherein the source domain style refers to a physical simulation style and uses One-Hot for coding; s7, outputting a generated image of the unmanned aerial vehicle physical simulation style and a mask image 2 of the unmanned aerial vehicle target area by the generator; s8, multiplying the generated image and the mask image 2 to obtain a foreground image, and multiplying the input synthesized image and the inverted mask image 2 to obtain a background image; S9, adding a foreground image representing the target area of the unmanned aerial vehicle and a background image outside the target area of the unmanned aerial vehicle to obtain a reconstructed image, wherein the reconstructed image represents the generated physical simulation style image of the unmanned aerial vehicle; S10, calculating reconstruction loss between the reconstructed unmanned aerial vehicle physical simulation image and the input unmanned aerial vehicle physical simulation image, and enhancing content consistency of the synthesized image and the input image; S11, inputting a synthetic image of the unmanned aerial vehicle target domain style and a real image of the unmanned aerial vehicle target domain style to a discriminator with a network structure PatchGAN; S12, outputting a discrimination result calculated for the input image by a discriminator, namely true/false, and domain classification results, namely source domain/sunny domain/cloudy domain/foggy domain; And S13, calculating the generation countermeasure loss between the judging result and the true value and the domain classification loss between the domain classification result and the true class.
3. The method for generating weather diversity based on generating visible light images of an opposing network unmanned aerial vehicle according to claim 2, wherein the reasoning process comprises the steps of: S21, acquiring an unmanned aerial vehicle image generated by computer modeling, performing preprocessing operation, embedding an unmanned aerial vehicle region in a real background image in a test data set, and recording the real background image as an unmanned aerial vehicle physical simulation image; s22, inputting an unmanned aerial vehicle physical simulation image and an One-Hot code of an unmanned aerial vehicle target domain style to a generator; s23, outputting a generated image of the unmanned aerial vehicle target domain style and a mask image 1 of the unmanned aerial vehicle target region by a generator; S24, multiplying the generated image and the mask image 1 to obtain a foreground image, and multiplying the input simulation image and the inverted mask image 1 to obtain a background image; And S25, adding the foreground image representing the unmanned aerial vehicle target area and the background image outside the unmanned aerial vehicle target area to obtain a composite image, wherein the composite image is the final output and represents the finally generated unmanned aerial vehicle target area style image, and the background is consistent with the input unmanned aerial vehicle physical simulation image.
4. The method for generating weather diversity of visible light image of unmanned aerial vehicle based on generation countermeasure network as claimed in claim 3, wherein in step S2, input image x from source domain is unmanned aerial vehicle physical simulation image, and target domain style code c is One-Hot code of real background image style, as input of generator G, the size is According to the size of the input image x Repeating to obtain a size of And spliced with x to finally obtain the vector with the size of I.e. the final input of generator G.
5. The method for generating weather diversity of visible light image of unmanned aerial vehicle based on generation countermeasure network of claim 4, wherein in steps S3-S5, generator G comprises 3-layer convolution layer downsampling, 6-layer residual block deep feature extraction, and deconvolution layer recovery resolution, and outputting comprises generating image And mask image The final composite image is expressed as: Wherein, the Representing a foreground image resulting from multiplication of the mask image with the generated image, The image is represented as a background image obtained by inverting the mask image and multiplying the mask image by the input image, and a final composite image is obtained by adding the foreground image and the background image.
6. The method for generating weather diversity based on visible images of an opposing network unmanned aerial vehicle of claim 5, wherein in steps S6-S9, the synthetic image of the unmanned aerial vehicle target domain style The output content, again together with the source domain style code c ', is taken as input to the generator G, which also includes a generated image G T (x ', c ') and a mask image G A (x ', c '), the generated image being multiplied by the mask image to obtain a foreground image, the input composite image being multiplied by the inverted mask image to obtain a background image, the foreground and background being added to obtain a final reconstructed image, expressed as: 。
7. The method for generating weather diversity of visible light image based on antagonistic network unmanned aerial vehicle according to claim 6, wherein in step S10, the image is reconstructed using a reconstruction loss constraint so as to be as close as possible to the initial input image x, by reducing the L 1 loss therebetween, the reconstruction loss being expressed as: where x represents the initial input image and, The composite image is represented by a representation of the composite image, Representing a reconstructed image generated by the composite image after it has passed through the generator again by calculating x and The L 1 loss in between, constrains the content of the reconstructed image.
8. The method for generating weather diversity based on visible light images of an unmanned aerial vehicle for generating countermeasure network according to claim 7, wherein in steps S11 and S12, the composite image and the real image are used as inputs of a discriminator D (patch gan), discrimination results and domain classification results are output, and constraint is performed by using the generation countermeasure loss and the domain classification loss, respectively; The discriminator comprises 6 convolution layers with the convolution kernel size of 4 and the step length of 2 and two convolution layers with the step length of 1, wherein the output of the last layer of convolution is simultaneously connected with two output layers, and one output is recorded as Participating in the calculation of the generated countermeasures against losses, another output being noted as Participate in the calculation of domain classification loss.
9. The method for generating weather diversity of visible light image of a countermeasure network unmanned aerial vehicle according to claim 8, wherein in step S13, a least squares loss is used to calculate a loss between the discrimination result and the true result to improve the quality and stability of the composite image, the loss being expressed as: Wherein the method comprises the steps of The representation generator accepts a real image X from field X and outputs a composite image according to the specified field label c, The result of the discrimination of the composite image is shown, Representing a discrimination result of the real image; the domain classification loss calculation method comprises the following steps: When optimizing the arbiter D, the domain classification penalty is defined as: Where x is the input image and where, Is a category of the source domain and, Representing that the domain classification result of the input image x obtained in the arbiter is the source domain Log represents the log operation; when optimizing generator G, the domain classification penalty is defined as: Where x is the input image and where, An image representing the input image converted to the target domain c by the generator, Representing the probability that the domain classification result obtained by the generated image in the discriminator is c; the total loss function of generator G and arbiter D is calculated as follows, wherein And Coefficients of domain classification loss and reconstruction loss, respectively: the reasoning process of the unmanned aerial vehicle visible light image weather diversity generation method only comprises the conversion from the source domain to the target domain in the generation part (the solid line flow direction in the generation part of fig. 4).
10. The method for generating visible light image weather diversity of unmanned aerial vehicle based on generation countermeasure network according to claim 9, wherein in step S21, preprocessing operation is performed on unmanned aerial vehicle images generated by computer modeling, the unmanned aerial vehicle regions are embedded into real background images in a test data set, and the image size is adjusted to 256×256 resolution and recorded as unmanned aerial vehicle physical simulation images; In steps S22 and S23, inputting the unmanned aerial vehicle physical simulation image x and the One-Hot code c of the unmanned aerial vehicle target domain style to a generator, and outputting the generated image of the unmanned aerial vehicle target domain style by the generator Mask image of target area of unmanned aerial vehicle ; In the steps S24 and S24, the generated image and the mask image are multiplied to obtain a foreground image, the input simulation image and the inverted mask image are multiplied to obtain a background image, the foreground image representing the target area of the unmanned aerial vehicle and the background image outside the target area of the unmanned aerial vehicle are added to obtain a synthesized image, the synthesized image is the final output, and the corresponding calculation formula is as follows: the composite image represents a final generated unmanned aerial vehicle target domain style image, wherein the background is consistent with the input unmanned aerial vehicle physical simulation image.

Description

Method for generating visible light image weather diversity of unmanned aerial vehicle based on generation countermeasure network Technical Field The invention belongs to the technical field of image generation, and relates to a visible light image weather diversity generation method based on a generation countermeasure network unmanned plane. Background The multiband photoelectric tracking system is a key component of the unmanned aerial vehicle detection defense system, and the visible light detection sensing module is combined with the servo cradle head, so that target locking and accurate tracking can be realized. The detection perception aiming at the unmanned aerial vehicle mainly utilizes a target detection algorithm to detect the photographed visible light image by the unmanned aerial vehicle, but in a highly complex air environment, the target detection algorithm is influenced by various factors such as light, weather and the like, and the recognition accuracy and tracking precision of the target detection algorithm are difficult to meet the requirements. Therefore, under different countermeasure scenes, how to develop a target detection algorithm with stronger adaptability has important significance for the research and development and the use of the multiband photoelectric tracking and aiming system. The traditional target detection algorithm generally relies on manual feature extraction and classical machine learning classifier, and the target detection algorithm based on deep learning mainly automatically learns the feature representation in the image through a neural network, so that the end-to-end mapping from the input image to the detection result is realized, and compared with the traditional method, the algorithm development and application process is simplified, and meanwhile, the detection performance of the algorithm is improved. However, the performance of the target detection algorithm based on deep learning is mainly dependent on the quantity and quality of training data, and in order to make the algorithm have better generalization to handle various complex environments, the training data needs to contain images with diversified styles. The generation countermeasure network is widely applied to the fields of image generation, style migration, image restoration and the like, wherein the style migration can acquire image translation capability by learning data of two different styles, such as a Pix2Pix network, a CycleGAN network, a StarGAN network and the like. The method is characterized in that CycleGAN is focused on unsupervised image translation, the problem that training data are difficult to acquire is solved by introducing cyclic consistency loss, and the idea of generating an countermeasure network by introducing conditions is introduced StarGAN, so that the task of simultaneously processing image translation in multiple fields is realized. Based on the advantages of the difficulty and style migration technology of the detection sensing unmanned aerial vehicle, the physical simulation unmanned aerial vehicle image generated by computer modeling can be combined with the background of the real world, and the physical simulation unmanned aerial vehicle image is converted into other real style images of various unmanned aerial vehicles through the style migration technology based on the generated countermeasure network, such as complex weather styles of sunny days, overcast days, foggy days and the like, so that the number of visible light images of the unmanned aerial vehicle is increased, and meanwhile, the image diversity is enriched, and the method can be used for training a target detection algorithm in a visible light detection sensing module, thereby improving the target locking performance of a multiband photoelectric tracking and aiming system. Disclosure of Invention Object of the invention The invention aims to improve the unmanned aerial vehicle locking capability of a visible light detection sensing module in a multiband photoelectric tracking system, a high-performance unmanned aerial vehicle target detection algorithm is needed, a large number of high-quality and various-style target images are needed to be used as training data during the training of the algorithm, the real image acquisition of the unmanned aerial vehicle is time-consuming and labor-consuming, the traditional method for enriching the data set through a physical simulation technology is generated through computer three-dimensional modeling, the unmanned aerial vehicle locking capability is characterized in that the appearance such as the type and the gesture of the unmanned aerial vehicle can be freely controlled and can be generated in batches, but the background diversity of the generated unmanned aerial vehicle physical simulation image is insufficient, the difference exists between the reality and the photographed unmanned aerial vehicle image, and the requirements of the quantity, the quality