CN-116630176-B - Image defogging method and device based on multiple attentives

CN116630176BCN 116630176 BCN116630176 BCN 116630176BCN-116630176-B

Abstract

The invention provides an image defogging method and device based on multiple attentions, and the method comprises the following steps of S1, obtaining an original foggy image, S2, extracting foggy features from the foggy image, S3, encoding the foggy features to obtain encoding features, S4, decoding the encoding features to obtain decoding features, S5, carrying out foggy residual decoding on the decoding features to obtain foggy residual, and S6, carrying out residual error on the original foggy image and the foggy residual to obtain a clear foggy-free image. The method can solve the problems of resolution loss, inaccurate characterization of fog characteristics and uneven defogging of the optical sensor in the defogging process of the fog map photographed by multiple scenes in the existing method.

Inventors

Tang Peiren
CHENG QI
LI JIE
GAO XIAOLI
WANG WEI
ZHAO HUOJUN
SONG CHENGCHENG
TAN LIGANG
YANG XIAOLI

Assignees

四川九洲电器集团有限责任公司

Dates

Publication Date: 20260508
Application Date: 20230321

Claims (3)

1. An image defogging method based on multiple attentiveness, which is characterized by comprising the following steps: s1, acquiring an original foggy image; s2, extracting haze characteristics from the fogged image; s3, calculating attention weights in three levels of a channel, a space and the self by adopting a multiple attention module, and weighting the haze characteristics in different levels according to the respective attention weights so as to obtain coding characteristics corresponding to a plurality of image blocks in a coding space; s4, sequentially decoding the coding features corresponding to the image blocks by adopting a self-attention decoder to obtain decoding features; S5, carrying out haze residual decoding on the decoding characteristics to obtain haze residues; S6, taking the original foggy image and the fog residue as residual errors to obtain a clear foggy image; in the step S2, the method for extracting the haze features from the hazed image comprises the following steps of: s21, sampling the foggy image at intervals of 1 by using a focus module to obtain four downsampled image blocks; S22, splicing the four downsampled image blocks according to the channel direction to obtain a downsampled feature map without resolution loss; S23, sending the downsampled feature map into a smooth expansion convolution network to obtain a smooth convolution feature; S24, splicing the smooth convolution characteristic and the downsampling characteristic graph along the channel direction to obtain a haze characteristic; step S3 comprises the following sub-steps: s31, the haze characteristics pass through a channel attention weighting layer to obtain channel attention weighted characteristics; S32, the channel attention weighted features pass through a space attention weighted layer to obtain space channel attention weighted features; S33, equally dividing the weighted characteristics of the attention of the spatial channels into a plurality of image blocks through a self-attention layer; s34, calculating a plurality of self-attentions of a plurality of image blocks through a multi-head attentions layer; s35, splicing the plurality of self-attentions, and then, obtaining coding features corresponding to the plurality of image blocks through an MLP residual error network; Step S4 comprises the following sub-steps: S41, decoding the coding feature corresponding to the first image block through a self-attention decoder to obtain a decoding feature corresponding to the first image block; S42, decoding the decoding characteristic corresponding to the first image block and the encoding characteristic corresponding to the second image block through a self-attention decoder together to obtain the decoding characteristic corresponding to the second image block; s43, decoding the coding features corresponding to all the image blocks in sequence according to the method of the step S42, and finally obtaining decoding features corresponding to all the image blocks; In step S5, the method for performing the haze residual decoding on the decoded feature to obtain the haze residual includes: mapping the decoding features corresponding to all image blocks of the original feature space to a target haze residual space through a multi-layer mixed convolution network, namely extracting haze residual, wherein the step S5 comprises the following sub-steps: s51, converting the decoding characteristics corresponding to all the image blocks into original sizes, and then splicing all the decoding characteristics according to the original positions of each image block to form a decoding characteristic; s52, decoding the composed decoding features to an original feature space through a dimension adaptation module composed of deconvolution layers; and S53, mapping the characteristic mapping of the original characteristic space to a target haze space through a haze residual mapping module formed by the convolution layers to obtain haze residues.
2. The multi-attention based image defogging method of claim 1, wherein in step S1, said original fogged image comprises: the image with fog to be processed is the image with fog obtained by the image sensor; and training the dataset, namely RESIDE datasets employed in the training phase, and resize the original RESIDE dataset to the size of the misted image to be processed.
3. A multi-attention based image defogging device for implementing the multi-attention based image defogging method of any of claims 1-2; The device comprises: The data acquisition module is used for acquiring an original foggy image; the feature extraction module is used for extracting haze features from the haze-containing image; The feature coding module is used for coding the haze features to obtain coding features; The feature decoding module is used for decoding the coding features to obtain decoding features; the haze residue decoding module is used for performing haze residue decoding on the decoding characteristics to obtain haze residues; And the residual error module is used for carrying out residual error on the original foggy image and the fog residues to obtain a clear foggy image.

Description

Image defogging method and device based on multiple attentives Technical Field The invention relates to the technical field of computer vision, in particular to an image defogging method and device based on multiple attentions. Background In recent years, with the rapid development of the deep learning technology and the wide application of the deep learning technology in the field of computer vision, the application of the deep learning technology related to the computer vision is becoming a hot spot of current research. Such as autopilot, border security, city monitoring, etc. In the application in the open environment, the imaging quality of the optical sensor is seriously affected by severe weather interference such as fog, haze, rain and the like, so that the implementation effect of the application is further affected. Therefore, how to remove the image mist and obtain the clear image is important. Aiming at the problem of image defogging, scholars propose a defogging method based on image enhancement, an algorithm based on image restoration and a method based on deep learning. Methods based on image enhancement have difficulty flexibly adapting to contrast changes in images and easily causing distortion of characteristics such as image color, contours, etc. Image restoration based algorithms have difficulty responding to large sky and white areas and can generate halos. The deep learning-based method can effectively solve the above problems, and thus has received a great deal of attention. Traditional deep learning algorithms have difficulty in accurately defogging images with flexibly changing scenes using simple convolutional layer stacks. In particular, there are several disadvantages: firstly, the original resolution loss is generated in the convolution stacking process; secondly, the haze characteristic extraction network is simple, and the haze characteristic is difficult to accurately represent; thirdly, the association condition of different areas of the image is not considered, so that defogging of each area is uneven. Disclosure of Invention The invention aims to provide an image defogging method and device based on multiple attentives, which are used for solving the problems of resolution loss, inaccurate characterization of fog characteristics and uneven defogging of an optical sensor in a defogging process of a fog map shot by multiple scenes in the conventional method. The invention provides an image defogging method based on multiple attentiveness, which comprises the following steps: s1, acquiring an original foggy image; s2, extracting haze characteristics from the fogged image; s3, coding the haze characteristics to obtain coding characteristics; s4, decoding the coding features to obtain decoding features; S5, carrying out haze residual decoding on the decoding characteristics to obtain haze residues; S6, taking the original foggy image and the fog residues as residual errors to obtain a clear foggy image. Further, in step S1, the original image with fog includes: the image with fog to be processed is the image with fog obtained by the image sensor; and training the dataset, namely RESIDE datasets employed in the training phase, and resize the original RESIDE dataset to the size of the misted image to be processed. Further, in step S2, the method for extracting haze features from a fogged image includes: Haze features are extracted from the foggy image through a focus feature extraction network based on smooth dilation convolution. Further, step S2 includes the following sub-steps: s21, sampling the foggy image at intervals of 1 by using a focus module to obtain four downsampled image blocks; S22, splicing the four downsampled image blocks according to the channel direction to obtain a downsampled feature map without resolution loss; S23, sending the downsampled feature map into a smooth expansion convolution network to obtain a smooth convolution feature; and S24, splicing the smooth convolution characteristic and the downsampling characteristic graph along the channel direction to obtain the haze characteristic. Further, in step S3, the method for encoding the haze feature to obtain the encoded feature includes: the attention weights are calculated in three levels of a channel, a space and the self by adopting a multi-attention module, and the haze characteristics are weighted according to the respective attention weights in different levels, so that the coding characteristics are obtained in the coding space. Further, step S3 includes the following sub-steps: s31, the haze characteristics pass through a channel attention weighting layer to obtain channel attention weighted characteristics; S32, the channel attention weighted features pass through a space attention weighted layer to obtain space channel attention weighted features; S33, equally dividing the weighted characteristics of the attention of the spatial channels into a plurality of image blocks through a self-attention