CN-115908827-B - Image denoising model processing method and device and electronic equipment

Abstract

According to the image denoising model processing method, the image denoising model processing device and the electronic equipment, when the student model is trained, the reconstruction loss value of the student model is calculated according to the edge characteristic difference of the processing results output by the teacher model and the student model, so that image denoising knowledge distillation based on the edge characteristic difference in the training process of the student model is realized, the student model can concentrate the learning range on the edge characteristic in the image, the learning range of the student model is reduced, and the improvement benefit brought by knowledge distillation learning is improved.

Inventors

• Luo Jiehao

Assignees

• 广州虎牙信息科技有限公司

Dates

Publication Date

20260512

Application Date

20221201

Claims (8)

1. 1. An image denoising model processing method, comprising: Obtaining a teacher model after training and a student model to be trained, wherein the teacher model and the student model are used for performing denoising processing on an image, and the structural complexity of the student model is smaller than that of the teacher model; acquiring a training sample, wherein the training sample comprises an image to be processed with noise and a reference image corresponding to the image to be processed and subjected to noise removal; Respectively inputting the images to be processed in the training sample into the teacher model and the student model to obtain a first processing result output by the teacher model and a second processing result output by the student model; extracting a first edge characteristic image corresponding to the image content edge position from the first processing result, and extracting a second edge characteristic image corresponding to the image content edge position from the second processing result; Calculating to obtain a difference feature image according to the difference value between each pixel point in the first edge feature image and the second edge feature image, carrying out pixel multiplication on the first processing result and the difference feature image to obtain a first weighted result, carrying out pixel multiplication on the second processing result and the difference feature image to obtain a second weighted result, and carrying out reconstruction loss calculation according to the first weighted result and the second weighted result to obtain a first reconstruction loss value; performing reconstruction loss calculation according to the reference image and the second processing result to obtain a second reconstruction loss value; and adjusting model parameters of the student model according to the first reconstruction loss value and the second reconstruction loss value.
2. 2. The method of claim 1, wherein the training sample further comprises a denoised reference image corresponding to the image to be processed, wherein the step of extracting a first edge feature image corresponding to an image content edge location from the first processing result and extracting a second edge feature image corresponding to an image content edge location from the second processing result comprises: Performing image content edge recognition on the reference image in the training sample to obtain edge contour position information of image content; And extracting and obtaining the first edge characteristic image from the first processing result according to the edge contour position information, and extracting and obtaining the second edge characteristic image from the second processing result according to the edge contour position information.
3. 3. The method of claim 2, wherein the edge profile position information comprises an edge profile mask image, wherein a value of a pixel corresponding to an edge position in the edge profile mask image is 1, and a value of a pixel corresponding to a non-edge position is 0; The step of extracting the first edge feature image from the first processing result according to the edge contour position information and extracting the second edge feature image from the second processing result according to the edge contour position information comprises the following steps: carrying out pixel multiplication on the first processing result and the edge contour mask image to obtain the first edge characteristic image; And carrying out pixel multiplication on the second processing result and the edge contour mask image to obtain the second edge characteristic image.
4. 4. The method according to claim 1, wherein the step of calculating a difference feature map according to a difference value between each pixel point in the first edge feature image and the second edge feature image includes: And calculating the absolute value of the difference value between each corresponding pixel point in the first edge characteristic image and the second edge characteristic image to obtain the difference characteristic image.
5. 5. The method according to claim 1, wherein the step of inputting the image to be processed in the training sample into the teacher model and the student model, respectively, to obtain a first processing result output by the teacher model and a second processing result output by the student model, comprises: Respectively inputting the images to be processed in the training sample into the teacher model and the student model, obtaining an image denoising result output by the teacher model as the first processing result, and obtaining an image denoising result output by the student model as the second processing result; or respectively inputting the images to be processed in the training sample into the teacher model and the student model, obtaining an intermediate feature image extracted when the teacher model executes image denoising as the first processing result, and obtaining an intermediate feature image extracted when the student model executes image denoising as the second processing result.
6. 6. An image denoising model processing device is characterized in that, the image denoising model processing device comprises: The model acquisition module is used for acquiring a teacher model after training and a student model to be trained, wherein the teacher model and the student model are used for performing denoising processing on an image, and the structural complexity of the student model is smaller than that of the teacher model; The sample acquisition module is used for acquiring a training sample, wherein the training sample comprises an image to be processed with noise and a reference image after noise is removed; The image processing module is used for inputting the images to be processed in the training sample into the teacher model and the student model respectively to obtain a first processing result output by the teacher model and a second processing result output by the student model; The edge feature extraction module is used for extracting a first edge feature image corresponding to the image content edge position from the first processing result and extracting a second edge feature image corresponding to the image content edge position from the second processing result; The loss calculation module is used for calculating and obtaining a difference characteristic image according to the difference value between each pixel point in the first edge characteristic image and each pixel point in the second edge characteristic image, carrying out pixel multiplication on the first processing result and the difference characteristic image to obtain a first weighted result, carrying out pixel multiplication on the second processing result and the difference characteristic image to obtain a second weighted result, and carrying out reconstruction loss calculation according to the first weighted result and the second weighted result to obtain a first reconstruction loss value; the loss calculation module is further used for carrying out reconstruction loss calculation according to the reference image and the second processing result to obtain a second reconstruction loss value; And the model adjustment module is used for adjusting the model parameters of the student model according to the first reconstruction loss value and the second reconstruction loss value.
7. 7. An electronic device comprising a processor and a machine-readable storage medium storing machine-executable instructions which, when executed by the processor, implement the method of any one of claims 1-5.
8. 8. A machine-readable storage medium storing machine-executable instructions which, when executed by one or more processors, implement the method of any one of claims 1-5.

Description

Image denoising model processing method and device and electronic equipment Technical Field The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for processing an image denoising model, and an electronic device. Background Images play an important role in human society, but significant noise exists in the images because the imaging, transmission and storage of the images are inevitably affected by some external factors. Due to the rapid development of the deep learning, the deep learning-related technology is widely applied to image denoising tasks, and achieves very remarkable effects. However, most of the image denoising algorithms with excellent performance are accompanied by huge network parameters, so that in order to obtain a faster reasoning speed, the algorithms are required to be deployed on equipment with excellent performance and high price, and the application cost of the algorithms is increased. Moreover, in some edge computing devices, such as cell phones, cameras, etc., and more limited by the computing and storage capabilities of the device, a large number of these parametric image denoising models cannot be deployed. The parameter quantity and the calculated quantity of the image denoising model are reduced, so that the deployment difficulty of the image denoising model can be greatly reduced, but the image denoising model with small parameter quantity is constructed, and the deployment cost can be reduced, the model reasoning speed can be improved, but the capability of the image denoising model is inevitably reduced. Therefore, in some image denoising schemes, a training model is used by a knowledge distillation method, and a teacher model with huge parameters and excellent image denoising capability is firstly constructed, then a student model with small parameters and relatively weak denoising capability is constructed, and the student model gradually fits the output of the teacher model in the training process, so that the performance of the student model gradually approaches to the teacher model. However, the knowledge distillation method applied to the image denoising field at present is basically based on the output of a complete feature map or a complete model, and the global distillation learning can lead a student model to lack a clear learning target, so that the improvement benefit brought by knowledge distillation learning is reduced. Disclosure of Invention In order to overcome the above-mentioned shortcomings in the prior art, an object of the present application is to provide an image denoising model processing method, which comprises: Obtaining a teacher model after training and a student model to be trained, wherein the teacher model and the student model are used for performing denoising processing on an image, and the structural complexity of the student model is smaller than that of the teacher model; Acquiring a training sample, wherein the training sample comprises an image to be processed which needs to have noise; Respectively inputting the images to be processed in the training sample into the teacher model and the student model to obtain a first processing result output by the teacher model and a second processing result output by the student model; extracting a first edge characteristic image corresponding to the image content edge position from the first processing result, and extracting a second edge characteristic image corresponding to the image content edge position from the second processing result; Calculating to obtain a first reconstruction loss value according to the difference between the first edge characteristic image and the second edge characteristic image; and adjusting model parameters of the student model according to the first reconstruction loss value. In one possible implementation manner, the training sample further comprises a reference image corresponding to the image to be processed after noise removal, and the method further comprises: performing reconstruction loss calculation according to the reference image and the second processing result to obtain a second reconstruction loss value; The step of adjusting model parameters of the student model according to the first reconstruction loss value includes: and adjusting model parameters of the student model according to the first reconstruction loss value and the second reconstruction loss value. In one possible implementation manner, the training sample further includes a reference image corresponding to the image to be processed after noise removal, the step of extracting a first edge feature image corresponding to an image content edge position from the first processing result and extracting a second edge feature image corresponding to an image content edge position from the second processing result includes: Performing image content edge recognition on the reference image in the training sample to obtain edge co