Search

EP-4216147-B1 - IMAGE PROCESSING METHOD AND APPARATUS, COMPUTER DEVICE, STORAGE MEDIUM, AND PROGRAM PRODUCT

EP4216147B1EP 4216147 B1EP4216147 B1EP 4216147B1EP-4216147-B1

Inventors

  • CHEN, JIAWEI
  • LI, Yuexiang
  • MA, KAI
  • ZHENG, YEFENG

Dates

Publication Date
20260513
Application Date
20220120

Claims (11)

  1. An image processing method, performed by a computer device, the method comprising: acquiring (202) a first image belonging to a first image domain, and inputting (202) the first image to an image processing model to be trained to obtain a second image belonging to a second image domain, wherein the image processing model is a generative network model in a generative adversarial network; extracting (204) a first image feature from the first image, and extracting (204) a second image feature from the second image, wherein the image feature comprises an image content feature and an image domain feature, the image content feature represents image content information for reflecting information of an object in the image, comprising a texture, a shape, or a spatial relationship, and the image domain feature represents image domain information for reflecting a style of the image, comprising a color or a brightness; calculating (206) a feature correlation degree characterizing a degree of similarity between the first image feature and the second image feature in the image content information; acquiring (208) first feature value distribution of the first image feature, and acquiring (208) second feature value distribution of the second image feature, wherein the first feature value distribution comprises occurrence probabilities of a plurality of first distribution values, and acquiring (208) the first feature value distribution of the first image feature comprises: performing (912A) statistics on feature values in the first image feature to obtain a first statistical value, wherein the first statistical value comprises a target mean value and a target standard deviation; acquiring (912B) standard probability distribution, and performing (912B) numerical value sampling based on the standard probability distribution to obtain numerical values that satisfy the standard probability distribution as target coefficients in a target coefficient set; scaling (912C) the target standard deviation based on the target coefficients to obtain target scaling values; performing (912C) translation processing on the target mean value based on the target scaling values to obtain target numerical values, the target numerical values corresponding to all the target coefficients forming a target numerical value set; determining (912D), based on a probability distribution relationship corresponding to the target numerical value set, a target occurrence probability corresponding to each of the target numerical values, the probability distribution relationship being a distribution to which the numerical values in the target numerical value set conform; and determining (912D) the target occurrence probability corresponding to each of the target numerical values as the occurrence probability of the first distribution value; calculating (210) a distribution correlation degree characterizing a degree of similarity between the first feature value distribution and the second feature value distribution, wherein an amount of image domain features contained in the first image feature and the second image feature is in negative correlation with the distribution correlation degree; and adjusting (212) model parameters of the image processing model to be trained to a direction in which the feature correlation degree is increased and a direction in which the distribution correlation degree is increased to obtain a trained image processing model, and processing an image using the trained image processing model.
  2. The method according to claim 1, wherein the feature correlation degree comprises a first feature discrimination probability, and wherein calculating (206) the feature correlation degree comprises: concatenating (910A) the first image feature and the second image feature, and determining (910A) a feature obtained through concatenation as a positive sample feature; and acquiring (910B) a feature discrimination model, and inputting (910B) the positive sample feature to the feature discrimination model to obtain the first feature discrimination probability; wherein adjusting (212) the model parameters of the image processing model to be trained to the direction in which the feature correlation degree is increased and the direction in which the distribution correlation degree is increased to obtain the trained image processing model comprises: obtaining a first model loss value based on the first feature discrimination probability, the first feature discrimination probability being in negative correlation with the first model loss value; obtaining a second model loss value based on the distribution correlation degree, the second model loss value being in negative correlation with the distribution correlation degree; and adjusting the model parameters of the image processing model to be trained based on the first model loss value and the second model loss value to obtain the trained image processing model.
  3. The method according to claim 2, further comprising: acquiring (910A) a third image in the second image domain, the third image being different from the second image; extracting (910A) a third image feature from the third image; and obtaining (910A) a negative sample feature based on a target image feature and the third image feature, and inputting (910B) the negative sample feature to the feature discrimination model to obtain a second feature discrimination probability, the target image feature being the first image feature or the second image feature; wherein obtaining the first model loss value based on the first feature discrimination probability comprises: obtaining (910C) the first model loss value based on the first feature discrimination probability and the second feature discrimination probability, the second feature discrimination probability being in positive correlation with the first model loss value.
  4. The method according to claim 3, further comprising: obtaining a model parameter adjustment gradient corresponding to the feature discrimination model based on the first model loss value; and adjusting model parameters of the feature discrimination model based on the model parameter adjustment gradient.
  5. The method according to any one of claims 1 to 4, further comprising: acquiring (902) a current image processing model, wherein the current image processing model is a generative network model trained at a current time in the generative adversarial network; inputting (904) a reference image to the current image processing model for processing to obtain a processed image, the reference image belonging to the first image domain; sending the processed image to a plurality of terminals, receiving deformation degrees of the processed image relative to the reference image from the plurality of terminals, and performing a weighted calculation on the deformation degrees to obtain a result of the weighted calculation as a target deformation degree; and in response to determining that the target deformation degree is greater than a deformation degree threshold, determining (908A) the current image processing model as the image processing model to be trained, and entering the operation of acquiring (202, 908B) the first image belonging to the first image domain; in response to determining that the target deformation degree is less than or equal to the deformation degree threshold, training (906B) the current image processing model based on a training image, and returning to the operation of inputting (904) the reference image to the current image processing model for processing; the training image belonging to the first image domain.
  6. The method according to any one of claims 1 to 5, wherein calculating (210) the distribution correlation degree comprises: acquiring a first distribution value from the first feature value distribution, and acquiring a second distribution value corresponding to the first distribution value from the second feature value distribution; performing difference calculation on the first distribution value and the second distribution value to obtain a difference value; and determining the distribution correlation degree based on the difference value, the distribution correlation degree being in negative correlation with the difference value.
  7. The method according to any one of claims 1 to 6, wherein the first image feature is obtained based on a first encoding model, and the second image feature is obtained based on a second encoding model; and wherein adjusting (212) the model parameters of the image processing model to be trained to the direction in which the feature correlation degree is increased and the direction in which the distribution correlation degree is increased to obtain the trained image processing model comprises: adjusting (916) model parameters of the image processing model to be trained, the first encoding model and the second encoding model to the direction in which the feature correlation degree is increased and the direction in which the distribution correlation degree is increased, to obtain an adjusted image processing model, first encoding model, and second encoding model; and returning to the operation of acquiring (202, 908B) the first image belonging to the first image domain, to continue model training until the image processing model to be trained converges to obtain (920) the trained image processing model.
  8. An image processing method, performed by a computer device, the method comprising: acquiring (702) an original image to be style-migrated in a first image domain; and inputting (708) the original image to the trained image processing model of claim 1 to generate a target image in a second image domain.
  9. A computer device, comprising a memory and one or more processors, the memory storing computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 7 or perform the method of claim 8.
  10. A computer-readable storage medium, storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 7 or perform the method of claim 8.
  11. A computer program product, comprising computer-readable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 7 or perform the method of claim 8.

Description

RELATED APPLICATION This application claims priority to Chinese Patent Application No. 202110154129.8, filed on February 4, 2021 and entitled "IMAGE PROCESSING METHOD AND APPARATUS, COMPUTER DEVICE, AND STORAGE MEDIUM". TECHNICAL FIELD This application relates to the field of image processing technologies, and in particular to an image processing method and apparatus, a computer device, a storage medium, and a program product. BACKGROUND With the development of artificial intelligence and multimedia technologies, there are more and more scenes in which users use image information in daily life and production activities. For example, users can perform domain conversion on images to obtain images in different image domains. For example, a sketch image can be converted into a two-dimensional image. At present, an image can be processed by artificial intelligence and using a machine learning model, and the image is input to the model to obtain a processed image. However, there is often a situation that the content of the processed image greatly changes from the content of the image which is not processed, for example, the converted image may be distorted, resulting in a poor image processing effect. SUMMARY According to various embodiments provided in this application, an image processing method and apparatus, a computer device, a storage medium and a program product are provided. Details of one or more embodiments of this application are provided in the accompany drawings and descriptions below. Other features, objectives, and advantages of this application become obvious with reference to the specification, the accompanying drawings, and the claims. The invention is set out in the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS To describe the technical solutions in the embodiments of this application more clearly, the following briefly introduces the accompanying drawings required for describing the embodiments. Apparently, the accompanying drawings in the following description show merely some embodiments of this application, and a person of ordinary skill in the art may still derive other drawings from these accompanying drawings without creative efforts. FIG. 1 is a diagram of an application environment of an image processing method in some embodiments;FIG. 2 is a schematic flowchart of an image processing method in some embodiments;FIG. 3 is a schematic diagram of a first image and a second image in some embodiments;FIG. 4 is a structural diagram of a generative adversarial network in some embodiments;FIG. 5 is a structural diagram of a cycle-consistent adversarial network in some embodiments;FIG. 6 is a schematic diagram of input of sample features to a feature discrimination model to obtain feature discrimination probabilities in some embodiments;FIG. 7 is a schematic flowchart of an image processing method in some embodiments;FIG. 8 is a diagram of an application environment of an image processing method in some embodiments;FIG. 9 is a schematic flowchart of an image processing method in some embodiments;FIG. 10 is a structural block diagram of an image processing apparatus in some embodiments;FIG. 11 is a structural block diagram of an image processing apparatus in some embodiments;FIG. 12 is an internal structural diagram of a computer device in some embodiments; andFIG. 13 is an internal structural diagram of a computer device in some embodiments. DETAILED DESCRIPTION To make objectives, technical solutions, and advantages of this application clearer and more understandable, this application is further described in detail below with reference to the accompanying drawings and the embodiments. It is to be understood that the specific embodiments described herein are only used for explaining this application, and are not used for limiting this application. An image processing method provided in this application may be applied to an application environment shown in FIG. 1. A terminal 102 communicates with a server 104 through a network. The terminal 102 may send an image domain migration request to the server 104, wherein the image domain migration request may carry an original image, a first image domain to which the original image belongs and a second image domain to which the original image is to be migrated, the server 104 may acquire a trained image processing model according to the first image domain and the second image domain, and the image processing model is used for performing image domain migration on an image in the first image domain to obtain an image in the second image domain. The server 104 may input the original image to the image processing model to generate a target image in the second image domain. The server 104 may return the generated target image to the terminal 102 or other terminals. The image processing model may be trained by the server 104. Specifically, the server 104 may obtain a first image belonging to the first image domain, input the first image to an image proce