CN-120339105-B - Mobile equipment image denoising method based on double-branch residual sparse network

CN120339105BCN 120339105 BCN120339105 BCN 120339105BCN-120339105-B

Abstract

The invention discloses a mobile equipment image denoising method based on a double-branch residual sparse network, which belongs to the technical field of image processing and comprises a residual sparse module, an attention guiding residual sparse module, a feature fusion module and a feature fusion module, wherein the residual sparse module captures local features in an image through mixed expansion convolution and residual connection, simultaneously reduces the parameter number and calculation complexity of a model, the attention guiding residual sparse module introduces a channel attention and pixel attention mechanism on the basis of the residual sparse module to adjust the weight of a feature map, focuses on an important area in the image, improves the denoising effect and the image quality, and the feature fusion module processes the added double-branch outputs through a residual module and combines the attention mechanism and an activation function to realize high-frequency detail retention and noise suppression. The method performs image denoising in resource-limited scenes such as unmanned aerial vehicles and the like, and solves the problems of large parameter quantity, low calculation efficiency, insufficient detail reservation and the like of the existing denoising model.

Inventors

GUO XIANGGUI
YIN ZHENLIANG

Assignees

北京科技大学顺德创新学院

Dates

Publication Date: 20260508
Application Date: 20250403

Claims (6)

1. The mobile equipment image denoising method based on the double-branch residual sparse network is characterized by comprising the following steps of: S1, data preparation and preprocessing: Shooting images in various scenes by using mobile equipment, carrying out data annotation and enhancement on the images, constructing a mobile equipment image data set, and preprocessing the images in the data set, wherein the preprocessing comprises normalization, graying, resolution adjustment and data segmentation; S2, constructing a double-branch residual sparse denoising network based on a series-parallel structure: The double-branch residual sparse denoising network comprises an upper branch network, a lower branch network and a feature fusion module, wherein the upper branch network is formed by connecting five residual sparse modules in series through up-down sampling and is used for gradually extracting image features and capturing multi-scale information; each residual sparse module comprises standard convolution and expansion convolution, and input features and output features are added through residual connection, so that the problem of gradient disappearance is avoided, and training stability is improved; the attention guiding residual sparse module increases channel attention and pixel attention mechanisms on the basis of the residual sparse module, so that a network is focused on a key region of an image, and the denoising effect is improved; The residual error sparse module and the attention guide residual error sparse module improve the receptive field by utilizing mixed expansion convolution without increasing the calculated amount, and the expansion convolution improves the receptive field and the feature extraction capability of the model under the condition of not increasing the calculated amount by introducing intervals in a convolution kernel, expanding the coverage range without increasing parameters, so that the model is light; the feature fusion module specifically comprises the following contents: the output of the upper branch network and the output of the lower branch network are added to be used as the input of a feature fusion module to carry out feature fusion; the feature fusion module consists of a standard convolution module, an attention guiding residual error sparse module, two standard convolutions and a Sigmoid activation function; the characteristic fusion process comprises the following steps: 1) The input of the feature fusion module sequentially passes through a standard convolution module, an attention guiding residual error sparse module and a first standard convolution; 2) Cascading the output with the normalized noise image, and sequentially convolving the output with a second standard through a Sigmoid activation function; 3) Multiplying the output of the first standard convolution with the output of the second standard convolution, and finally, carrying out residual error with the original noise image to obtain a final denoised image; the function of the feature fusion process is expressed as follows: Wherein f rb represents the input of the RB module, RB represents the feature fusion module, cat represents the cascading operation, C represents the standard convolution, I n represents the input noise image, and Sig represents the Sigmoid function; Representing the output of the upper branch network of the deep feature extraction module; the method comprises the steps of representing the output of a lower branch network of a deep feature extraction module, CBR representing a standard convolution module, ARSB representing a concentration guidance residual error sparse module; s3, designing a loss function L of a double-branch residual sparse denoising network, wherein the loss function L is as follows: Wherein, the And Respectively representing the denoised image of the model and the corresponding clean image, wherein N represents the total number of the images; s4, dividing the mobile equipment image obtained in the S1 into a training set and a testing set through data segmentation, inputting the training set image into a constructed dual-branch residual sparse denoising network, and calculating the gradient of a loss function by using a back propagation method until the loss function tends to be stable, so as to obtain a trained dual-branch residual sparse denoising network model; S5, inputting the noise image in the test set into a trained double-branch residual sparse denoising network to obtain a denoising image.
2. The mobile device image denoising method based on the dual-branch residual sparse network according to claim 1, wherein the construction method of the upper-branch network in S2 specifically comprises: 1) A standard convolution layer with the convolution kernel size of 3 multiplied by 3 and the step length of 1, batch normalization and ReLU activation functions are used for forming a standard convolution module; 2) An expansion convolution layer with a convolution kernel size of 3 multiplied by 3 and a step length of 1 and expansion ratios of 2 and 3 respectively, and a batch normalization and ReLU activation function are used for forming an expansion convolution module; 3) The standard convolution module, the expansion convolution module with the expansion ratio of 2, the standard convolution module and the expansion convolution module with the expansion ratio of 3 are sequentially connected and connected with an input as residual errors to form a residual error sparse module; 4) Sequentially connecting five residual sparse modules, connecting the first three residual sparse modules through downsampling, and connecting the last three residual sparse modules through upsampling to form an upper branch network; The function of the upper branch network is expressed as follows: Wherein, the Representing the output of each RSB module, i E1, 2,3,4,5, O shallow representing the input of the upper branch network, RSB representing the residual sparse module, down and Up representing downsampling and upsampling, respectively; Representing the output of the upper branch network of the deep feature extraction module.
3. The mobile device image denoising method based on the dual-branch residual sparse network according to claim 2, wherein the residual sparse module in the upper branch network expands the receptive field of the upper branch network by combining standard convolution and expansion convolution operation to capture multi-scale characteristic information, and the specific contents are as follows: firstly, extracting image features through a standard convolution layer, and then, increasing a receptive field by using expansion convolution to obtain wider context information on the premise of ensuring calculation efficiency; The residual connection is applied between the input and the output of the module, is used for relieving the gradient vanishing problem in the deep network, promoting the information flow, accelerating the training and convergence of the network, and the function is expressed as follows: Wherein f rsb represents the input of the RSB module, CBR represents the standard convolution module, DBR 2 represents the expansion convolution module with the expansion ratio of 2, DBR 3 represents the expansion convolution module with the expansion ratio of 3, the number of channels of the standard convolution and the expansion convolution is 64, and the convolution kernel is 3 multiplied by 3.
4. The lower branch network of the mobile device image denoising method based on the dual-branch residual sparse network according to claim 1, wherein the method for constructing the lower branch network in S2 specifically comprises: 1) Connecting a standard convolution module with an expansion rate of 2, and performing residual connection to form an internal residual sparse module; 2) The internal residual error sparse module, the channel attention mechanism, the pixel attention mechanism, the standard convolution module and the expansion convolution module with the expansion ratio of 3 are sequentially connected and connected with an input as residual error, so that an attention guiding residual error sparse module is formed; 3) Sequentially connecting the five attention-directed residual sparse modules to form a lower branch network; The function of the lower branch network is expressed as follows: Wherein, the Representing the output of each ARSB module, i.e. 1,2,3,4,5, O shallow representing the input of the upper branch network; representing the output of the lower branch network of the deep feature extraction module.
5. The mobile device image denoising method based on the dual-branch residual sparse network according to claim 4, wherein the attention-directed residual sparse module in the lower-branch network specifically comprises the following contents: Firstly, sequentially connecting a standard convolution module and an expansion convolution module with expansion rate of 2, and forming an internal residual sparse module through residual connection; The internal residual error sparse module is sequentially connected with a channel attention mechanism, a pixel attention mechanism, a standard convolution module and an expansion convolution module with the expansion rate of 3 to form an attention guiding residual error sparse module, wherein in the attention guiding residual error sparse module, the channel and the pixel attention mechanism adaptively allocate weights for different characteristics, so that the attention of a network to important characteristics is improved; And finally, adding the output of the module with the initial input through residual connection to ensure the efficient flow of information and strengthen gradient propagation in the training process, wherein the attention-guided residual sparse module combines multi-scale feature extraction and attention mechanism with residual connection to improve the performance and stability of a lower branch network in complex tasks, and the functions are expressed as follows: where f arsb denotes the input of the ARSB module, CAB denotes the channel attention module, and PAB denotes the pixel attention module.
6. The mobile device image denoising method based on the dual-branch residual sparse network according to claim 1, wherein the step of calculating the gradient of the loss function by using the back propagation method in S4 is performed on the model, and the specific implementation steps are as follows: 1) Setting a dual-branch residual sparse denoising network as a training mode; 2) Inputting the noise image in the training set into a double-branch residual sparse denoising network, carrying out forward propagation, calculating and outputting a fusion denoising image; 3) Calculating the loss between the model output and the original clean image according to the loss function; 4) Calculating the gradient of the loss function according to a back propagation method; 5) Using an Adam optimizer, updating parameters of a model according to gradient change of a loss function, wherein the learning rate of the optimizer is set to be 0.001, and the super parameter beta 1 =0.9,β 2 =0.99; 6) Repeating the above process until the loss function is stable, and obtaining the trained double-branch residual sparse denoising network model.

Description

Mobile equipment image denoising method based on double-branch residual sparse network Technical Field The invention relates to the technical field of image processing, in particular to an image denoising method of a double-branch residual sparse network based on a serial-parallel structure, which is suitable for covering mobile equipment with limited resources such as unmanned aerial vehicles, mobile robots and the like. Background Image denoising is an important task in the field of image processing, aimed at reducing or eliminating noise in an image, while preserving the details and features of the image as much as possible. With the rapid development of mobile communication technology and digital imaging sensors, mobile phones and cameras have become the most commonly used photographing devices in daily life, and the demand for high quality images has been increasing. Therefore, it is important to develop a denoising technique that can effectively process a noise image at low computational cost and achieve good visual reproduction quality. In this context, image denoising techniques have made significant progress. For example, the Block matching and 3D Filtering (Block-MATCHING AND 3D Filtering) algorithm produces a sharper image by grouping similar image blocks, applying 3D Filtering to reduce noise, and averaging the results, while preserving detail. BM3D-Net convolutional neural networks inspired by BM3D algorithms are modeled by developing BM3D computational flows and introducing "extraction" and "aggregation" layers. In addition, there are other various denoising algorithms, such as a weighted kernel norm minimization (Weighted Nuclear Norm Minimization) algorithm, a multichannel WNNM algorithm, a block sparse collaborative low-rank algorithm based on sparse and collaborative low-rank matrix decomposition, a three-dimensional magnetic resonance imaging denoising model, and the like. Although traditional denoising models perform well under certain conditions, they are often limited by the large number of parameters. The denoising model based on the deep neural network improves performance by learning complex features and modes, generally requires fewer super parameters and provides faster reasoning speed, thereby achieving better effects in various denoising tasks. For example, the denoising convolutional neural network (Deep Convolutional Neural Network) improves denoising performance by learning noise distribution in an image and residuals in the denoising process. The fast and flexible denoising network (Fast and Flexible Denoising Network) achieves efficient image denoising with lower computational cost and smaller model size. However, as the depth of the network increases, the number of parameters and the operating time also increase, which makes the model difficult to deploy in practical applications. To address this problem, researchers have enhanced the representation capabilities of denoising networks by widening the network architecture, such as the double convolutional neural network (Deep Convolutional Neural Network) and the batch normalized depth network model (Batch Renormalization Deep Network). In addition, attention mechanisms are increasingly playing a role in image denoising, which enables models to better understand image content, eliminate noise, and improve image quality and definition. Although existing deep learning networks perform well in terms of image denoising, there are some drawbacks. For a larger network model, although the denoising effect is better, the hardware requirement on equipment is higher, the running time is long, and the memory occupation is large. While for smaller network models, the operating speed and space occupation can meet the equipment requirements, the denoising performance is relatively insufficient. Therefore, it is a challenging direction to study a network model that meets both performance requirements and has less complexity. Disclosure of Invention The invention aims to provide a mobile equipment image denoising method based on a double-branch residual sparse network, which is used for solving the problems that denoising performance and model complexity are difficult to balance, are difficult to expand on small mobile equipment, cannot be well suitable for various image processing scenes and the like in the prior art. In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: a mobile device image denoising method based on a double-branch residual sparse network comprises the following steps: S1, data preparation and preprocessing: Shooting images in various scenes by using mobile equipment, carrying out data annotation and enhancement on the images, constructing a mobile equipment image data set, and preprocessing the images in the data set, wherein the preprocessing comprises normalization, graying, resolution adjustment and data segmentation; S2, constructing a double-branch residual sparse denoi