CN-115293966-B - Face image reconstruction method, device and storage medium

CN115293966BCN 115293966 BCN115293966 BCN 115293966BCN-115293966-B

Abstract

The invention provides a face image reconstruction method, a face image reconstruction device and a storage medium, which belong to the field of image reconstruction, wherein the method comprises the steps of sequentially carrying out super-resolution image training analysis on a super-resolution model to be trained through each original face image to obtain a super-resolution training model; the method comprises the steps of respectively identifying images of original face images through a super-resolution training model to obtain target super-resolution images, carrying out training analysis on face reconstruction images of a face reconstruction model to be trained through all the original face images and all the target super-resolution images to obtain a face reconstruction training model, and carrying out image reconstruction on the target super-resolution images through the face reconstruction training model to obtain a face image reconstruction result. The invention can reconstruct the HR face image with high fidelity and identity perception, can extract more detailed information and improves the visual fidelity.

Inventors

LU TAO
WANG YIYI
CHENG FANGFANG
ZHANG YANDUO
FANG WENHUA

Assignees

武汉工程大学

Dates

Publication Date: 20260505
Application Date: 20220627

Claims (9)

1. The face image reconstruction method is characterized by comprising the following steps of: Importing a plurality of original face images, constructing a super-resolution model to be trained, and sequentially carrying out super-resolution image training analysis on the super-resolution model to be trained through each original face image to obtain a super-resolution training model corresponding to each original face image; Respectively carrying out image recognition on each original face image through a super-resolution training model corresponding to each original face image to obtain a target super-resolution image corresponding to each original face image; Constructing a face reconstruction model to be trained, and performing training analysis on face reconstruction images on the face reconstruction model to be trained through all original face images and all target super-resolution images to obtain a face reconstruction training model; respectively carrying out image reconstruction on each target super-resolution image through the face reconstruction training model to obtain face reconstruction images corresponding to each original face image, and taking all the face reconstruction images as face image reconstruction results; The super-resolution model to be trained comprises a pair of up-sampling blocks which are sequentially arranged and a pair of down-sampling blocks which are sequentially arranged, the super-resolution model to be trained is sequentially subjected to training analysis of super-resolution images through the original face images, and the process of obtaining the super-resolution training model corresponding to the original face images comprises the following steps: s11, sequentially carrying out up-sampling analysis on each original face image through the up-sampling block to obtain up-sampled face images corresponding to each original face image; S12, sequentially carrying out downsampling analysis on each original face image through the downsampling block to obtain an original face super-resolution image corresponding to each original face image; S13, sequentially carrying out image enhancement processing on each original face super-resolution image through a plurality of preset filters to obtain a positive sample set corresponding to each original face image; S14, sequentially carrying out degradation treatment on each original face super-resolution image through a plurality of preset filters to obtain a negative sample set corresponding to each original face image; S15, analyzing the comparison learning loss values of each original face super-resolution image, a positive sample set corresponding to the original face image and a negative sample set corresponding to the original face image in sequence to obtain the comparison learning loss values corresponding to each original face image; and S16, judging whether each comparison learning loss value is larger than a preset first loss threshold value in sequence, if so, carrying out parameter updating on the super-resolution model to be trained according to the comparison learning loss values, taking the updated super-resolution model to be trained as a super-resolution model to be trained corresponding to the original face image, returning to the step S11, and if not, taking the super-resolution model to be trained as a super-resolution training model corresponding to the original face image.
2. The face image reconstruction method according to claim 1, wherein the upsampling block comprises a plurality of sequentially arranged residual channel attention blocks, a first Convolution layer, pixel recombination layer A convolution layer; The process of step S11 includes: Sequentially amplifying the original face images through a plurality of residual channel attention blocks to obtain amplified face images corresponding to the original face images; Through the first part The convolution layer sequentially carries out first feature extraction on each amplified face image to obtain first feature extracted face images corresponding to each original face image; sequentially carrying out up-sampling treatment on each face image subjected to primary feature extraction through the pixel recombination layer to obtain face images to be feature extracted corresponding to each original face image; through the said And the convolution layer sequentially carries out feature extraction again on each face image to be subjected to feature extraction to obtain up-sampled face images corresponding to each original face image.
3. The face image reconstruction method according to claim 1, wherein the downsampling block comprises a second Convolutional layer, leakyReLU activate function layer and third A convolution layer; The process of step S12 includes: Through the second part The convolution layer sequentially carries out third feature extraction on each face image after up-sampling to obtain a face image after third feature extraction corresponding to each original face image; performing first mapping processing on each face image after the third feature extraction sequentially through the LeakyReLU activation function layer to obtain mapped face images corresponding to each original face image; Through the third part And the convolution layer sequentially carries out fourth feature extraction on each mapped face image to obtain an original face super-resolution image corresponding to each original face image.
4. The face image reconstruction method according to claim 1, wherein the process of step S13 includes: Sequentially performing image enhancement processing on each original face super-resolution image through a plurality of preset filters and a first formula to obtain a positive sample set corresponding to each original face image, wherein the first formula is as follows: , Wherein, the Is the first A positive sample set corresponding to the super-resolution image of the original face, Is the first The original face super-resolution image, RF is the image enhancement operation, Is the first And P is the number of positive sample images.
5. The face image reconstruction method according to claim 1, wherein the process of step S14 includes: Sequentially performing degradation processing on each original face super-resolution image through a plurality of preset filters and a second formula to obtain a negative sample set corresponding to each original face image, wherein the second formula is as follows: , Wherein, the Is the first A negative sample set corresponding to the super-resolution images of the individual original faces, Is the first The super-resolution image of the original face is obtained, In order to reduce the quality of the product, Is the first And N is the number of negative sample images.
6. The face image reconstruction method according to claim 1, wherein the process of step S15 includes: sequentially taking each original face super-resolution image and a positive sample set corresponding to each original face image as a positive image set corresponding to each original face image; sequentially taking each original face super-resolution image and a negative sample set corresponding to each original face image as a negative pair image set corresponding to each original face image; Extracting feature images of the positive facing image sets and the negative facing image sets corresponding to the original face images sequentially by utilizing a feature extraction network to obtain positive facing feature images corresponding to the original face images and negative facing feature images corresponding to the original face images; Calculating a contrast learning loss value of each positive feature map, an original face super-resolution image corresponding to each original face image, a positive sample set corresponding to each original face image, a negative feature map corresponding to each original face image and a negative sample set corresponding to each original face image in sequence according to a third formula to obtain the contrast learning loss value corresponding to each original face image, wherein the third formula is as follows: , Wherein, the In order to compare the values of the learning losses, In order to be in direct opposition to the feature map, Is a negative pair of feature maps, and the feature maps, Is an original super-resolution image of the human face, As a set of positive samples, As a set of negative examples of the sample, As a result of the desired value(s), And Are super parameters.
7. The face image reconstruction method according to claim 1, wherein the process of performing training analysis on the face reconstruction image of the face reconstruction model to be trained by using all original face images and all target super-resolution images to obtain a face reconstruction training model comprises: S31, extracting feature vectors of the original face images and the target super-resolution images through a deep convolutional neural network to obtain original face feature vectors corresponding to the original face images and target super-resolution image feature vectors corresponding to the target super-resolution images; S32, carrying out regularization treatment on original face feature vectors corresponding to the original face images and target super-resolution image feature vectors corresponding to the target super-resolution images by using an L2 regularization algorithm to obtain regularized face images corresponding to the original face images and regularized super-resolution images corresponding to the original face images; S33, calculating identity loss values of all regularized face images and all regularized super-resolution images through a fourth formula to obtain the identity loss values, wherein the fourth formula is as follows: , Wherein, the In order to be an identity loss value, Is the first Regularized face images corresponding to the original face images, Is the first Regularized super-resolution images corresponding to the original face images, The number of the original face images; And S34, judging whether the identity loss value is larger than a preset second loss threshold value, if so, carrying out parameter updating on the face reconstruction model to be trained according to the identity loss value, taking the updated face reconstruction model to be trained as the face reconstruction model to be trained, returning to the step S31, and if not, taking the face reconstruction model to be trained as the face reconstruction training model.
8. A face image reconstruction apparatus, comprising: The first training analysis module is used for importing a plurality of original face images, constructing a super-resolution model to be trained, and sequentially carrying out training analysis on the super-resolution image on the super-resolution model to be trained through each original face image to obtain a super-resolution training model corresponding to each original face image; The image recognition module is used for respectively carrying out image recognition on each original face image through a super-resolution training model corresponding to each original face image to obtain a target super-resolution image corresponding to each original face image; The second training analysis module is used for constructing a face reconstruction model to be trained, and carrying out training analysis on the face reconstruction image of the face reconstruction model to be trained through all original face images and all target super-resolution images to obtain a face reconstruction training model; The face image reconstruction result module is used for respectively carrying out image reconstruction on each target super-resolution image through the face reconstruction training model to obtain face reconstruction images corresponding to each original face image, and taking all the face reconstruction images as face image reconstruction results; The super-resolution model to be trained comprises a pair of up-sampling blocks and a pair of down-sampling blocks, wherein the up-sampling blocks are sequentially arranged, and the first training analysis module is specifically used for: s11, sequentially carrying out up-sampling analysis on each original face image through the up-sampling block to obtain up-sampled face images corresponding to each original face image; S12, sequentially carrying out downsampling analysis on each original face image through the downsampling block to obtain an original face super-resolution image corresponding to each original face image; S13, sequentially carrying out image enhancement processing on each original face super-resolution image through a plurality of preset filters to obtain a positive sample set corresponding to each original face image; S14, sequentially carrying out degradation treatment on each original face super-resolution image through a plurality of preset filters to obtain a negative sample set corresponding to each original face image; S15, analyzing the comparison learning loss values of each original face super-resolution image, a positive sample set corresponding to the original face image and a negative sample set corresponding to the original face image in sequence to obtain the comparison learning loss values corresponding to each original face image; and S16, judging whether each comparison learning loss value is larger than a preset first loss threshold value in sequence, if so, carrying out parameter updating on the super-resolution model to be trained according to the comparison learning loss values, taking the updated super-resolution model to be trained as a super-resolution model to be trained corresponding to the original face image, returning to the step S11, and if not, taking the super-resolution model to be trained as a super-resolution training model corresponding to the original face image.
9. A computer readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the face image reconstruction method according to any one of claims 1 to 7.

Description

Face image reconstruction method, device and storage medium Technical Field The invention mainly relates to the technical field of image reconstruction, in particular to a face image reconstruction method, a face image reconstruction device and a storage medium. Background In recent years, a contrast learning method based on an instance discrimination pre-task has been widely used in unsupervised representation learning, which learns visual representations by making views from the same instance similar and views from different instances dissimilar, the learned visual representations can be applied to various downstream tasks, particularly advanced tasks such as image clustering, knowledge distillation, supervised image classification, and the like, and desired results can be obtained. The principle of contrast learning is to pull the positive samples closer to the anchor point and push the negative samples away in the representation space, thereby increasing the mutual information in the learned representation. The choice of positive and negative samples depends on the particular downstream task, e.g., consider enhancement of the original data as positive samples, or consider multiple views of the same sample as positive samples. When low-level image processing tasks are involved, there are some challenges to directly applying contrast learning methods. First, the learned global visual representation is not suitable for low-level tasks that require rich texture and context information. Second, while a series of data enhancements have been proposed to generate positive and negative samples for advanced downstream tasks, most complex data enhancements, except for some simple geometric enhancements, do not maintain dense pixel correspondence and are therefore not suitable for low-level tasks. Third, contrast loss requires a meaningful embedding space, and lower-level tasks aim to reconstruct the recovered results in the data space, as compared to higher-level tasks that attempt to obtain the best semantic representation. It is important to explore a suitable and meaningful embedding space in which contrast loss can be efficiently defined. Current contrast learning-based methods for low-level tasks are mainly focused on using negative samples, while taking real images as positive samples. For example, wu et al treat degraded images as negative samples and propose a novel image defogging method with contrast regularization. Wang et al use other examples of data sets as negative examples for image super resolution and underwater image restoration. These approaches demonstrate the effectiveness of incorporating contrast constraints into low-level tasks. Another direction of investigation is to model the statistical features of the image by contrast learning. Dong et al assume that two image blocks from the same sample have similar noise distributions and two image blocks from two different samples have two different noise distributions, and propose a residual contrast penalty for joint demosaicing and denoising. Wang et al apply contrast loss to pre-train a kernel estimation model that aims to separate different degradation and obtain different degradation sense representations. Zhang et al propose a contrast learning strategy in the feature channel space to obtain the feature with constant resolution. They take as a sample the feature maps of the different channels and assume that the corresponding channels of the LR and HR feature maps are positive, while the feature maps from the different channels are negative. In these methods, one direction of investigation is that a positive sample is defined as the original image, while a negative sample is simply defined as the future image or other image in the dataset. Although these negative samples are different from the reconstructed image, they are easily distinguished, i.e. they are too far apart to contribute to contrast loss. While another research direction has attempted to generate some noise-immune global features of the image based on contrast learning, these approaches ignore positive and negative samples that are effective in constructing reconstructed images. Furthermore, since the contrast penalty of these methods is defined over some specific embedding space, it does not scale well to other methods. The result of the current deep learning-based face super-resolution method tends to be smooth and uncertain, look unnatural, unreliable, and ignore the special texture of the face. Because the mapping from LR to HR cannot be uniquely determined, there may be multiple function spaces, so the superdivision result may be an average of all possible outputs of the SR network. Disclosure of Invention The invention aims to solve the technical problem of providing a face image reconstruction method, a face image reconstruction device and a storage medium aiming at the defects of the prior art. The technical scheme for solving the technical problems is as follows, the