CN-116681622-B - Dynamic scene image deblurring method, system and equipment

CN116681622BCN 116681622 BCN116681622 BCN 116681622BCN-116681622-B

Abstract

The invention provides a dynamic scene image deblurring method, a system and equipment, which concretely comprise the following steps of constructing a dynamic scene image deblurring network, comprising a multi-scale dense feature extraction module connected between U-Net network convolution layers, a ConvLSTM bidirectional communication structure inserted between U-Net network adjacent jump connections and a U-Net network structure optimization strategy inserted in a U-Net network encoder and decoder, constructing a total loss function, training the dynamic scene image deblurring network to obtain a dynamic scene image deblurring network model, inputting an image to be processed into the dynamic scene image deblurring network model to obtain a deblurred image, and the method can effectively reduce the problems of texture detail loss, noise suppression, ringing artifact prevention and the like in the image recovery process, thereby facilitating the development of tasks and works required subsequently.

Inventors

LIU GUANGHUI
YANG QI
MENG YUEBO
GAO JIAHAO
LI CHAO
XU SHENGJUN

Assignees

西安建筑科技大学

Dates

Publication Date: 20260505
Application Date: 20230626

Claims (8)

1. A method for deblurring a dynamic scene image is characterized by comprising the following specific steps: S1, constructing a dynamic scene image deblurring network, wherein the dynamic scene image deblurring network comprises a multi-scale dense feature extraction module connected between U-Net network convolution layers, convLSTM bidirectional communication structures inserted between adjacent jump connections of the U-Net network and U-Net network structure optimization strategies inserted in U-Net network encoders and decoders; s2, constructing a total loss function, and training a dynamic scene image deblurring network to obtain a dynamic scene image deblurring network model; S3, inputting the image to be processed into a dynamic scene image deblurring network model to obtain a deblurred image; in S1, a U-Net network structure optimization strategy comprises a hole convolution and a sub-pixel convolution, wherein the hole convolution is used for downsampling in an encoder, and the sub-pixel convolution is used for upsampling in a decoder; In S1, the multi-scale dense feature extraction module includes a dense feature extraction network and a multi-level pyramid pooling branch SPP, where the dense feature extraction network is used to obtain deep features of an image, the multi-level pyramid pooling branch SPP is used to obtain multi-scale local detail features, and the deep features and the multi-scale local detail features are fused to obtain global detail features for optimizing the definition of a blurred image.
2. The method of claim 1, wherein the dense feature extraction network comprises convolution layers in an original U-Net structure interconnected in a feedforward manner, and dense short connection is adopted between the convolution layers closer to an input layer and an output layer, wherein the input of a first t th convolution layer in the dense feature extraction network is that T is the subsequent layer number of the dense connection structure, and the output is as follows: (1) Wherein the method comprises the steps of For 3 consecutive operations, i.e. batch normalization, linear units and one 3 x 3 convolution.
3. A method of deblurring a dynamic scene image according to claim 1, wherein the multi-level pyramid pooling branch SPP comprises three parallel 3 x 3 dilated convolution capture blocks with dilation ratios of 2,4, 8, respectively.
4. The method as set forth in claim 1, wherein ConvLSTM bi-directional communication structures include forward paths and reverse paths distributed between adjacent hop connections of the U-Net network, the forward paths being from bottom to top, and the reverse paths being from top to bottom, a convolution long short-term memory model ConvLSTM being interposed between the forward paths and the reverse paths for extracting multi-dimensional feature information from adjacent feature context relationships and fusing the multi-dimensional feature information with the current feature of the decoding part.
5. The method of claim 4, wherein the forward path and the reverse path between adjacent hop connections of the U-Net network are paired, the forward path on the side near the U-Net network encoder and the reverse path on the side near the U-Net network decoder.
6. A method for deblurring a dynamic scene image as recited in claim 1, wherein in S2, the total loss function Including the L 2 loss function and the L p perceived loss, the formulas are as follows: (2) wherein λ represents a weight typically λ=0.01; assuming that there is a training set (B i ,I i ) comprising For blurred/sharp images, the L 2 loss function formula is as follows: (3) Wherein, the F is a dynamic scene image deblurring network with parameters; The L p perceptual loss formula is as follows: (4) Wherein: J is the corresponding clear original image; And Respectively represent Characteristic diagrams corresponding to J, C j ,H j and W j respectively represent And The number of channels, height and width of the feature map.
7. A dynamic scene image deblurring system applying the method of any of claims 1-6, comprising: The network construction module is used for constructing a dynamic scene image deblurring network and comprises a multi-scale dense feature extraction module connected between U-Net network convolution layers, convLSTM bidirectional communication structures inserted between adjacent jump connections of the U-Net network and U-Net network structure optimization strategies inserted in a U-Net network encoder and decoder; the network training module is used for constructing a total loss function, training the dynamic scene image deblurring network and obtaining a dynamic scene image deblurring network model; And the image processing module is used for inputting the image to be processed into the dynamic scene image deblurring network model to obtain a deblurred image.
8. Terminal equipment, characterized in that it comprises a processor, a memory and a computer program stored in said memory and executable on said processor, said processor implementing the method according to any one of claims 1-6 when executing said computer program or the functions of the modules in the system according to claim 7 when said processor executes said computer program.

Description

Dynamic scene image deblurring method, system and equipment Technical Field The invention belongs to the technical field of dynamic scene deblurring, and particularly relates to a method, a system and equipment for deblurring a dynamic scene image. Background The image is a main carrier for people to acquire information, and is also an important symbol term in the process of information exchange and culture transfer in various industries in the current society. Along with the improvement of social life quality and the actual needs of daily work, related technologies based on images are rapidly developed, such as target detection, image segmentation and target tracking, however, in the image acquisition process, due to multiple interference of external environments, the images tend to have motion blur, the motion blur of the images not only affects the perception of people, but also seriously reduces the processing efficiency of the technologies such as subsequent target detection, image recognition and the like, so how to remove the motion blur of the images and restore clear images gradually becomes an important research direction in the field of computer vision. The dynamic scene deblurring method generally accurately describes the original characteristic information of the image in the image deblurring process by using a common method based on constraint least square filtering, a regularization method, a deep learning method and the like, and improves the processing efficiency of the technologies such as subsequent target detection, image recognition and the like. In recent years, due to the fact that deep learning depends on the strong feature representation capability, time and quality can be simultaneously considered in the description process of image features compared with the traditional manual feature extraction method. However, the existing motion blur removing network has the problems that texture details are lost, noise cannot be restrained, ringing artifacts are generated and the like easily in some image recovery processes, and a plurality of difficulties are brought to subsequent tasks and works to be carried out. In order to solve the problems of unclear object edge, poor visual perception and the like caused by dynamic blurring, a series of motion blurring removal applications are developed by a plurality of researchers by means of the characteristics of high-low semantic information fusion, high training speed and the like of a U-Net structure. Although the method proposed by the U-Net encoding and decoding structure can achieve a good image deblurring effect, only a small part of characteristic information can be captured in a characteristic extraction stage, global context semantic information can not be obtained effectively, the extracted characteristics have the problems of low utilization efficiency, serious loss and the like generally, and the obtained image also has the image quality problems of detail texture deletion and the like. Some students acquire image information of different scales by using a dense multi-acceptance domain feature information extraction and aggregation related method, so that feature multiplexing efficiency is increased, and the effectiveness of the application in improving the detail expressive force of the blurred image is verified. Although the image definition is effectively restored by deep feature extraction and global feature aggregation on the basis of dense connection, the edge and detail parts of the image are greatly improved, feature information extracted by a network in an encoding stage cannot be efficiently transmitted to a decoding stage, and a small amount of ringing artifacts still exist in the image easily, so that information such as characters, slogans and the like are difficult to distinguish. Disclosure of Invention In order to solve the problems in the prior art, the invention provides a method, a system and equipment for deblurring a dynamic scene image, which can effectively reduce the problems of losing texture details, suppressing noise to a certain extent, preventing ringing artifacts and the like in the image recovery process, and is convenient for the development of tasks and works required subsequently. In order to achieve the above purpose, the invention provides the following technical scheme that the method for deblurring the dynamic scene image comprises the following specific steps: S1, constructing a dynamic scene image deblurring network, wherein the dynamic scene image deblurring network comprises a multi-scale dense feature extraction module connected between U-Net network convolution layers, convLSTM bidirectional communication structures inserted between adjacent jump connections of the U-Net network and U-Net network structure optimization strategies inserted in U-Net network encoders and decoders; s2, constructing a total loss function, and training a dynamic scene image deblurring network to obtain a dynamic sc