Search

CN-117291794-B - Method for solving false edges and checkerboard by improving enhanced network based on MBLLEN

CN117291794BCN 117291794 BCN117291794 BCN 117291794BCN-117291794-B

Abstract

The application provides a method for solving false edges and checkerboards based on MBLLEN improved reinforced networks, which comprises the steps of S1, downsampling structures, S2, downsampling structure optimization, S3, secondary downsampling, S4, upsampling structure selection, S5, upsampling structure optimization, S6, connection of context feature map information, S7, secondary upsampling, and S8, connection of context feature map information again. Because MBLLEN has the defects of the network structure, the defects of false edges and checkerboard effect on the image enhancement of a general real scene are avoided, the rigid defects of the original structure are overcome by the modification of the application, the occurrence of the false edges and the checkerboard is avoided, the expression capability of the network is enhanced, and the application can well converge under the condition that the exposure interval of a training set is not fixed. The feature map is reduced, the calculation force required by the model is reduced, the context information is connected, and the information loss of a simple linear structure is avoided.

Inventors

  • WEI YOUQI

Assignees

  • 合肥君正科技有限公司

Dates

Publication Date
20260512
Application Date
20220616

Claims (7)

  1. 1. A method for improving an enhanced network to solve false edges and checkerboards based on MBLLEN, the method comprising the steps of: S1, selecting a downsampling structure: aiming at a denoising part in a dim light enhancement task, selecting an average pooling structure with denoising property as a downsampling structure, wherein the downsampling multiple is 2, and the size of a feature map is reduced by half; s2, optimizing a downsampling structure: the characteristic of enhancing the expression capacity through 1*1 convolution constitutes an optimized structure; s3, secondary downsampling: the method comprises the steps of performing secondary downsampling on an image, wherein the size of a feature map of the image is changed into 1/4 of that of the original image, and the secondary downsampling is performed by connecting a structure of high-frequency information maximum pooling and low-frequency information average pooling; S4, upsampling structure selection: selecting bilinear interpolation, wherein the up-sampling multiple is 2; s5, up-sampling structure optimization: adding 3*3 and 1*1 convolutions before and after the optimized structure in the step S2; S6, connecting the context feature map information, and splicing the up-sampling result in the S5 with the down-sampling result in the S2; S7, secondary up-sampling: Again, the same steps as those of the step S5 are carried out; And S8, connecting the context feature map information again, and splicing the secondary up-sampling result with the result which is not subjected to down-sampling, namely splicing the original image which is not subjected to S1 with the result of S7.
  2. 2. The method for improving the solution of false edges and checkerboards based on MBLLEN of the enhanced network as claimed in claim 1, wherein said composing the optimized structure in step S2 includes: s2.1, a first convolution layer with a convolution kernel of 3 and an output channel of 64 is processed; s2.2, an average pooling layer is subjected to 2 times downsampling; s2.3, passing through a second convolution layer with a convolution kernel of 1 and an output channel of 32.
  3. 3. The method for solving false edges and checkerboards based on MBLLEN improved enhancement networks as claimed in claim 1, wherein said step S3 of subsampling the image comprises: s3.1,3 x 64 convolving conv; s3.2, respectively carrying out maximum pooling and average pooling: s3.2.1, 2-fold maximum pooling MaxPool; performing a1 x 32 convolution conv; s3.2.2, 2-fold average pooling AvgPool; performing a1 x 32 convolution conv; And S3.3, splicing the concat, wherein the feature images are three-dimensional matrixes (h, w and c), h represents length, w represents width, c represents a channel, and the splicing is that two feature images with the same h and w are spliced together in the dimension c.
  4. 4. The method for improving a solution to false edges and checkerboards based on MBLLEN of claim 1, wherein the adding of the convolution of 3*3 and 1*1 in step S5 on the basis of the optimized structure of step S2 includes: s5.1, a first convolution layer with a convolution kernel of 3 and an output channel of 64 is processed; s5.2, passing through a second convolution layer with a convolution kernel of 3 and an output channel of 64; S5.3, an average pooling layer is subjected to 2 times downsampling; S5.4, passing through a third convolution layer with a convolution kernel of 1 and an output channel of 32; S5.5, passing through a fourth convolution layer with a convolution kernel of 1 and an output channel of 32.
  5. 5. The method for solving false edges and checkerboards based on MBLLEN improved enhanced network of claim 1, wherein said step S7 includes; S7.0, selecting an up-sampling structure, namely selecting bilinear interpolation, wherein the up-sampling multiple is 2; s7.1, performing 3*3 convolution conv; s7.2, performing a 3x 64 convolution conv; S7.3, carrying out 2-time average pooling AvgPool; S7.4, performing a 1x 32 convolution conv; s7.5, performing 1*1 convolution conv.
  6. 6. The method for solving false edges and checkerboard based on MBLLEN improved enhanced network as claimed in claim 1, wherein in the step S8, the original image without S1 is spliced with the result of S7, and the method comprises the steps of enabling feature images to be three-dimensional matrixes (h, w and c), wherein h represents length, w represents width, c represents channel, and splicing two feature images with the same h and w together in the dimension of c.
  7. 7. The method for solving false edges and checkerboard by using MBLLEN-based improved enhancement network as claimed in claim 1, wherein said steps S6 and S8 use the self-adaptive capacity of the model itself to obtain the necessary information of the effect that the model is expected to achieve in the tag data set, said self-adaptive capacity means that the deep learning supervised learning gradually reduces the difference between the model output and the tag data set according to the minimized model output and the loss function of the tag data set in the model training, and trains out the weight information adapting to the change, namely the self-adaptive capacity, said necessary information includes brightness, contrast and edge information.

Description

Method for solving false edges and checkerboard by improving enhanced network based on MBLLEN Technical Field The invention relates to the technical field of low light enhancement in a deep learning neural network low-level visual task, in particular to a method for solving false edges and checkerboards based on MBLLEN improved enhancement networks. Background The existing supervised deep learning low light enhancement has very direct positive or negative influence on the result due to the fact that the input and the output are pictures, and most of models are mainly U-shaped networks or other full convolution structures. In real applications, the input and output images are usually 1080p or even 4k high-definition images, so that the requirement on computational power is extremely high, so that the operations of downsampling operation and upsampling recovery are necessary, and common downsampling is performed by step and fill (padding) of convolution or by pooling operation. Upsampling is often achieved by means of deconvolution or interpolation. The effect of selecting a different structure will be apparent visually, since the output of the model is also a picture. The MBLLEN network's native enhancement structure chooses to use padding (padding) for valid mild downsampling, and then upsampling is performed by deconvolution. However, the operation of filling in valid by convolution is used for downsampling, and the original image size is slightly smaller than the original feature map, so that the aim of obviously reducing the calculation amount cannot be achieved. On the other hand, the feature map in this situation is up-sampled, so that obvious false edges appear, and even though the feature map can be improved by training, the defects existing in the structure cannot be perfectly solved. The up-sampling is carried out by using the deconvolution, the calculation mode of the deconvolution has the defect of causing a chessboard effect, and the effect of a final image can be greatly influenced even if training improvement is not carried out, so that the quality of a dim light image after enhancement is seriously reduced, and the requirement of engineering application in a real scene can not be met. Furthermore, the common terminology in the prior art is as follows: Bilinear interpolation, which is a linear interpolation extension of interpolation functions with two variables in mathematics, has the core idea of performing linear interpolation once in two directions respectively, and is widely applied to aspects such as signal processing, digital image processing, video processing and the like. Checkerboard effect assuming that 1 black object is contained in the deconvoluted image, the pixel color of the black object portion should be smoothly transitioned. Or, at the extreme, the body part should be entirely black. In the actually generated image, the part is formed by a near-black square block with a depth and a shallow depth, much like a network of chessboard. This is the so-called checkerboard effect. The false edges are obvious marks on the edges of the image, and if the images are combined after being segmented, the phenomenon of grid shape is often presented. Downsampling, which is a process of shrinking the feature map, and upsampling, which is a process of enlarging the feature map. Disclosure of Invention In order to solve the problems, the application aims to construct a new enhanced network branch structure based on MBLLEN, solve the false edge and checkerboard effect of the network, enhance the expression capability of the network and better promote the denoising and model effect. Specifically, the invention provides a method for solving false edges and checkerboards based on MBLLEN improved enhancement networks, which comprises the following steps: s1, selecting a downsampling structure: For a denoising part in a dim light enhancement task, selecting average pooling with denoising property as a downsampling structure, wherein the downsampling multiple is 2, and the size of a feature map is reduced by half through the operation; s2, optimizing a downsampling structure: the characteristic of enhancing the expression capacity through 1*1 convolution constitutes an optimized structure; s3, secondary downsampling: The image is subjected to secondary downsampling, and the size of a feature map of the image is changed into 1/4 of the original size; s4, up-sampling structure selection: Selecting bilinear interpolation, wherein the upsampling multiple is 2; S5, optimizing an up-sampling structure: the operation is to add 3*3 and 1*1 convolutions before and after the optimized structure in the step S2; s6, connecting the context feature map information, and splicing the up-sampling result in the S5 with the down-sampling result in the S2; S7, secondary up-sampling: Again, the same steps as those of the step S5 are carried out; s8, the context feature map information is connected again, and the secondary